Hi all,
\nI just made a guide of how to use conv net to achieve NLL of 1.382285. On a modern GPU, it only takes 5 minute to get this result.
\nPlease check https://github.com/antinucleon/cxxnet/tree/master/example/kaggle_bowl , and don't forget to star CXXNET :)
\n\nCXXNET is yet another neural network toolkit. It is able to run on NVIDIA GPU or CPU. If you want to run on CPU, you only need OpenCV & MKL(or OpenBlas/ATLAS) installed. No other dependence is required.
\nPS. I made it in a hurry, so if there is any problem please let me know!
", "messages": [ { "id": "60426", "postDate": "12/17/2014 04:10:34", "content": "Hi all,
\nI just made a guide of how to use conv net to achieve NLL of 1.382285. On a modern GPU, it only takes 5 minute to get this result.
\nPlease check https://github.com/antinucleon/cxxnet/tree/master/example/kaggle_bowl , and don't forget to star CXXNET :)
\n\nCXXNET is yet another neural network toolkit. It is able to run on NVIDIA GPU or CPU. If you want to run on CPU, you only need OpenCV & MKL(or OpenBlas/ATLAS) installed. No other dependence is required.
\nPS. I made it in a hurry, so if there is any problem please let me know!
", "rawMarkdown": "", "votes": null }, { "id": "60429", "postDate": "12/17/2014 05:03:41", "content": "Can I use CXXNET on a windows machine?
", "rawMarkdown": "", "votes": null }, { "id": "60431", "postDate": "12/17/2014 05:06:18", "content": "I don't have Windows machine with GPU so I am not very sure.. I guess you can but you need to configure by yourself.
\n[quote=kartal;60429]
\nCan I use CXXNET on a windows machine?
\n[/quote]
", "rawMarkdown": "", "votes": null }, { "id": "60439", "postDate": "12/17/2014 07:20:29", "content": "Thank you for your share.
\nBut I want to know, if I can run CXXNET with CPU?
\nI just have a macbook with intel GPU...
", "rawMarkdown": "", "votes": null }, { "id": "60440", "postDate": "12/17/2014 07:31:27", "content": "You can, you need blas or MKL installed. Then set dev=cpu in config file
\n\n[quote=Issac;60439]
\nThank you for your share.
\nBut I want to know, if I can run CXXNET with CPU?
\nI just have a macbook with intel GPU...
\n[/quote]
", "rawMarkdown": "", "votes": null }, { "id": "60441", "postDate": "12/17/2014 07:46:28", "content": "Very nice, star-ed it on github!
", "rawMarkdown": "", "votes": null }, { "id": "60464", "postDate": "12/17/2014 11:13:08", "content": "Thank you very much! How much main memory and GPU memory does it require?
", "rawMarkdown": "", "votes": null }, { "id": "60483", "postDate": "12/17/2014 14:12:56", "content": "What's the difference between cxxnet and Caffe?
", "rawMarkdown": "", "votes": null }, { "id": "60499", "postDate": "12/17/2014 16:30:48", "content": "[quote=Bing Xu;60426]
\nHi all,
\nI just made a guide of how to use conv net to achieve NLL of 1.382285. On a modern GPU, it only takes 5 minute to get this result.
\nPlease check https://github.com/antinucleon/cxxnet/tree/master/example/kaggle_bowl , and don't forget to star CXXNET :)
\nPS. I made it in a hurry, so if there is any problem please let me know!
\n[/quote]
\nBing, thanks! This is great work, as usual :-)
\nI hate to tell you this, but in the beginning I'm afraid you'll end up getting more questions about how to install this (and errors while installing) than how to use it. I think adding more details in the wiki about the install, dependencies, where to get the dependencies, how to make sure those work, how to troubleshoot errors,.. might save you time in the long run.
\nBut again, awesome work! Thanks a lot!
", "rawMarkdown": "", "votes": null }, { "id": "60503", "postDate": "12/17/2014 16:43:42", "content": "Ahh... I wish I had GPU :-/
", "rawMarkdown": "", "votes": null }, { "id": "61418", "postDate": "12/17/2014 20:02:17", "content": "Thanks Bing - this is as good and straight-forward of an intro to deep learning using a GPU as anyone could hope for. I understand you had a GTX 780M or something like that for processing. May I also ask how much RAM your system had, and how much was in use while processing, if you happen to know? Thanks!
", "rawMarkdown": "", "votes": null }, { "id": "61419", "postDate": "12/17/2014 20:09:22", "content": "300MB GPU mem, 1GB CPU mem is enough. I remember
\n[quote=Johnny Strings;61418]
\nThanks Bing - this is as good and straight-forward of an intro to deep learning using a GPU as anyone could hope for. I understand you had a GTX 780M or something like that for processing. May I also ask how much RAM your system had, and how much was in use while processing, if you happen to know? Thanks!
\n[/quote]
", "rawMarkdown": "", "votes": null }, { "id": "61420", "postDate": "12/17/2014 20:09:52", "content": "You can use CPU. MKL will give you reasonable performance
\n[quote=Abhishek;60503]
\nAhh... I wish I had GPU :-/
\n[/quote]
", "rawMarkdown": "", "votes": null }, { "id": "61421", "postDate": "12/17/2014 20:10:50", "content": "Thanks for this advice, I will put link of wiki to README
\n[quote=Giulio;60499]
\n[quote=Bing Xu;60426]
\nHi all,
\nI just made a guide of how to use conv net to achieve NLL of 1.382285. On a modern GPU, it only takes 5 minute to get this result.
\nPlease check https://github.com/antinucleon/cxxnet/tree/master/example/kaggle_bowl , and don't forget to star CXXNET :)
\nPS. I made it in a hurry, so if there is any problem please let me know!
\n[/quote]
\nBing, thanks! This is great work, as usual :-)
\nI hate to tell you this, but in the beginning I'm afraid you'll end up getting more questions about how to install this (and errors while installing) than how to use it. I think adding more details in the wiki about the install, dependencies, where to get the dependencies, how to make sure those work, how to troubleshoot errors,.. might save you time in the long run.
\nBut again, awesome work! Thanks a lot!
\n[/quote]
", "rawMarkdown": "", "votes": null }, { "id": "61423", "postDate": "12/17/2014 20:56:13", "content": "thanks for sharing. i'm attempting to get this setup with caffe, so this will be a good guide. i'll share my version too if i get it working :)
", "rawMarkdown": "", "votes": null }, { "id": "61426", "postDate": "12/17/2014 21:21:27", "content": "I am one of the author. In short, they are similar. CXXNET is much shorter, and concise in terms of lines of code.
\n\n[quote=ziyuang;60483]
\nWhat's the difference between cxxnet and Caffe?
\n[/quote]
", "rawMarkdown": "", "votes": null }, { "id": "61427", "postDate": "12/17/2014 21:22:31", "content": "[quote=Abhishek;60503]
\nAhh... I wish I had GPU :-/
\n[/quote]
\nHave you considered using AWS GPU instances? http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using_cluster_computing.html
If you're okay with spot pricing and sudden cutoffs, I'm sure you can get a good rate.
[quote=Tianqi Chen;61426]
\nI am one of the author. In short, they are similar. CXXNET is much shorter, and concise in terms of lines of code.
\n[quote=ziyuang;60483]
\nWhat's the difference between cxxnet and Caffe?
\n[/quote]
\n[/quote]
\nHow about from the user's angle? Does it mean CXXNET will be faster, or does it mean it will be easier to learn because of the conciseness?
", "rawMarkdown": "", "votes": null }, { "id": "61432", "postDate": "12/17/2014 22:09:44", "content": "Can CXXNET be run on Macbook pro with GPU?
\nI'm getting this:
\nIn file included from ./mshadow/mshadow/tensor.h:11:
./mshadow/mshadow/tensor_base.h:76:12: fatal error: 'mkl.h' file not found
#include <mkl.h>
^
1 error generated.
make: *** [cxxnet_data.o] Error 1
In default it is using MKL. I don't have Macbook so I am not sure. Ideally it is able to run on all NVIDIA GPU.
\nPlease refer https://github.com/antinucleon/cxxnet#build-guide to build it.
\n[quote=clustifier;61432]
\nCan CXXNET be run on Macbook pro with GPU?
\nI'm getting this:
\nIn file included from ./mshadow/mshadow/tensor.h:11:
./mshadow/mshadow/tensor_base.h:76:12: fatal error: 'mkl.h' file not found
#include <mkl.h>
^
1 error generated.
make: *** [cxxnet_data.o] Error 1
[/quote]
", "rawMarkdown": "", "votes": null }, { "id": "61436", "postDate": "12/17/2014 23:36:42", "content": "Thanks for sharing the code. I have a problems when installing cxxnet on Ubuntu 14.04
\n$ build.sh blas=1
\ng++ -Wall -g -O3 -msse3 -Wno-unknown-pragmas -funroll-loops -I./mshadow/ -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_CBLAS=1 -o bin/cxxnet cxxnet/cxxnet_main.cpp cxxnet_data.o cxxnet_nnet_cpu.o cxxnet_nnet_gpu.o -lm -lcudart -lcublas -lcurand -lz `pkg-config --libs opencv` -lblas
/usr/bin/ld: cxxnet_data.o: undefined reference to symbol 'sem_post@@GLIBC_2.2.5'
//lib/x86_64-linux-gnu/libpthread.so.0: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
make: *** [bin/cxxnet] Error 1
How could I fix it?
\n\nMeanwhile, do you mean add 'dev=cpu' to bowl.conf ? I didn't find 'dev=cpu' or 'dev=gpu' in any fil
\ne. I guess that you are using GPU and it is the default option.
\n\nMany Thanks
\n\n[
\nquote=Bing Xu;60440]
\nYou can, you need blas or MKL installed. Then set dev=cpu in config file
\n[quote=Issac;60439]
\nThank you for your share.
\nBut I want to know, if I can run CXXNET with CPU?
\nI just have a macbook with intel GPU...
\n[/quote]
\n[/quote]
", "rawMarkdown": "", "votes": null }, { "id": "61437", "postDate": "12/17/2014 23:40:12", "content": "I suggest you post in issue on Github about building problem. It is about your pthread problem on your machine. I guess may be upgrade your gcc will help. But I am not sure because I never use Mac.
\n\n\nYou can add ```dev = cpu``` at global setting part, I mean no indent
\n\n[quote=Jason;61436]
\nThanks for sharing the code. I have a problems when installing cxxnet on Ubuntu 14.04
\n$ build.sh blas=1
\ng++ -Wall -g -O3 -msse3 -Wno-unknown-pragmas -funroll-loops -I./mshadow/ -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_CBLAS=1 -o bin/cxxnet cxxnet/cxxnet_main.cpp cxxnet_data.o cxxnet_nnet_cpu.o cxxnet_nnet_gpu.o -lm -lcudart -lcublas -lcurand -lz `pkg-config --libs opencv` -lblas
/usr/bin/ld: cxxnet_data.o: undefined reference to symbol 'sem_post@@GLIBC_2.2.5'
//lib/x86_64-linux-gnu/libpthread.so.0: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
make: *** [bin/cxxnet] Error 1
How could I fix it?
\nMeanwhile, do you mean add 'dev=cpu' to bowl.conf ? I didn't find 'dev=cpu' or 'dev=gpu' in any fil
\ne. I guess that you are using GPU and it is the default option.
\nMany Thanks
\n[
\nquote=Bing Xu;60440]
\nYou can, you need blas or MKL installed. Then set dev=cpu in config file
\n[quote=Issac;60439]
\nThank you for your share.
\nBut I want to know, if I can run CXXNET with CPU?
\nI just have a macbook with intel GPU...
\n[/quote]
\n[/quote]
\n[/quote]
", "rawMarkdown": "", "votes": null }, { "id": "61438", "postDate": "12/18/2014 00:25:02", "content": "Many Thanks Bing,
\nI give an updated details of my building environment at GitHub issue 14
\nCheers.
\n\n[quote=Bing Xu;61437]
\nI suggest you post in issue on Github about building problem. It is about your pthread problem on your machine. I guess may be upgrade your gcc will help. But I am not sure because I never use Mac.
\nYou can add ```dev = cpu``` at global setting part, I mean no indent
\n[quote=Jason;61436]
\nThanks for sharing the code. I have a problems when installing cxxnet on Ubuntu 14.04
\n$ build.sh blas=1
\ng++ -Wall -g -O3 -msse3 -Wno-unknown-pragmas -funroll-loops -I./mshadow/ -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_CBLAS=1 -o bin/cxxnet cxxnet/cxxnet_main.cpp cxxnet_data.o cxxnet_nnet_cpu.o cxxnet_nnet_gpu.o -lm -lcudart -lcublas -lcurand -lz `pkg-config --libs opencv` -lblas
/usr/bin/ld: cxxnet_data.o: undefined reference to symbol 'sem_post@@GLIBC_2.2.5'
//lib/x86_64-linux-gnu/libpthread.so.0: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
make: *** [bin/cxxnet] Error 1
How could I fix it?
\nMeanwhile, do you mean add 'dev=cpu' to bowl.conf ? I didn't find 'dev=cpu' or 'dev=gpu' in any fil
\ne. I guess that you are using GPU and it is the default option.
\nMany Thanks
\n[
\nquote=Bing Xu;60440]
\nYou can, you need blas or MKL installed. Then set dev=cpu in config file
\n[quote=Issac;60439]
\nThank you for your share.
\nBut I want to know, if I can run CXXNET with CPU?
\nI just have a macbook with intel GPU...
\n[/quote]
\n[/quote]
\n[/quote]
\n[/quote]
", "rawMarkdown": "", "votes": null }, { "id": "61442", "postDate": "12/18/2014 03:23:38", "content": "Hi Bing,
\nWhen I run the gen_train.py function I am getting these warnings "convert is not recognized as an internal or external command". Are you using a special python package to resize images?
", "rawMarkdown": "", "votes": null }, { "id": "61443", "postDate": "12/18/2014 03:25:08", "content": "You need install http://www.imagemagick.org/ to use convert
\n[quote=kartal;61442]
\nHi Bing,
\nWhen I run the gen_train.py function I am getting these warnings "convert is not recognized as an internal or external command". Are you using a special python package to resize images?
\n[/quote]
", "rawMarkdown": "", "votes": null }, { "id": "61446", "postDate": "12/18/2014 04:39:41", "content": "OK thanks I did get gen_train.py to work but when I look at the converted images the original sizes are preserved. They are not 48x48 as suggested.
", "rawMarkdown": "", "votes": null }, { "id": "61447", "postDate": "12/18/2014 05:07:59", "content": "OK I figured out the problem. Maybe this is due to I am using MinGW in Windows but the correct form of the python code should be as below in order for the resizing to work.
\n\nimport os
import sys
import subprocess
if len(sys.argv) < 3:
print "Usage: python gen_train.py input_folder output_folder"
exit(1)
fi = sys.argv[1]
fo = sys.argv[2]
cmd = "convert "
classes = os.listdir(fi)
os.chdir(fo)
for cls in classes:
try:
os.mkdir(cls)
except:
pass
imgs = os.listdir(fi + cls)
for img in imgs:
md = ""
md += cmd
md += fi + cls + "/" + img
md += " -resize 48x48!"
md += " " + fo + cls + "/" + img
os.system(md)
Thanks Bing, I just got it working and submitted results.
\nFor others - I'm using Ubuntu 14.04, and had to muck around with the CXXNet Makefiles a bit, adding -I and -L references to the CUDA library locations.
", "rawMarkdown": "", "votes": null }, { "id": "61497", "postDate": "12/18/2014 17:12:16", "content": "[quote=Bing Xu;60426]
\nHi all,
\nI just made a guide of how to use conv net to achieve NLL of 1.382285.
\n[/quote]
\n\nBefore I try to install it, can it be expected to work on an NVIDIA GRID K520? This is an older model with compute capability 3.0.
", "rawMarkdown": "", "votes": null }, { "id": "61503", "postDate": "12/18/2014 20:45:31", "content": "Hi, Bing,
\nI follow your net config file and create a caffe proto file as attached, but it cannot get your result. Is there any difference between your cxxnet and caffe?
", "rawMarkdown": "", "votes": null }, { "id": "61522", "postDate": "12/19/2014 06:10:19", "content": "Hi Bing,
\nAfter compiling successfully for cpu and adding the dev=cpu I'm getting this:
\n$ ../../bin/cxxnet bowl.conf
CXXNetTrainer, devCPU=1
ConvolutionLayer: nstep=256
ConvolutionLayer: nstep=256
ConvolutionLayer: nstep=128
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
node[0].shape: 256,3,48,48
node[1].shape: 256,96,12,12
node[2].shape: 256,96,12,12
node[3].shape: 256,96,6,6
node[4].shape: 256,128,8,8
node[5].shape: 256,128,8,8
node[6].shape: 256,128,8,8
node[7].shape: 256,128,8,8
node[8].shape: 256,128,4,4
node[9].shape: 1,1,256,2048
node[10].shape: 1,1,256,512
node[11].shape: 1,1,256,512
node[12].shape: 1,1,256,512
node[13].shape: 1,1,256,512
node[14].shape: 1,1,256,121
ThreadImagePageIterator:image_list=./train.lst, bin=./train.bin
cannot find models/image_mean.bin: create mean image, this will take some time...
[ 9000] images processed, 1 sec elapsedError:decoding fail
Can you please see what I might do wrong?
\nThanks,
\nC
", "rawMarkdown": "", "votes": null }, { "id": "61523", "postDate": "12/19/2014 06:12:01", "content": "It means opencv can't decode image in the binary file. I suggest you check your opencv and process in generating binary file.
\n[quote=clustifier;61522]
\nHi Bing,
\nAfter compiling successfully for cpu and adding the dev=cpu I'm getting this:
\n$ ../../bin/cxxnet bowl.conf
CXXNetTrainer, devCPU=1
ConvolutionLayer: nstep=256
ConvolutionLayer: nstep=256
ConvolutionLayer: nstep=128
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
node[0].shape: 256,3,48,48
node[1].shape: 256,96,12,12
node[2].shape: 256,96,12,12
node[3].shape: 256,96,6,6
node[4].shape: 256,128,8,8
node[5].shape: 256,128,8,8
node[6].shape: 256,128,8,8
node[7].shape: 256,128,8,8
node[8].shape: 256,128,4,4
node[9].shape: 1,1,256,2048
node[10].shape: 1,1,256,512
node[11].shape: 1,1,256,512
node[12].shape: 1,1,256,512
node[13].shape: 1,1,256,512
node[14].shape: 1,1,256,121
ThreadImagePageIterator:image_list=./train.lst, bin=./train.bin
cannot find models/image_mean.bin: create mean image, this will take some time...
[ 9000] images processed, 1 sec elapsedError:decoding fail
Can you please see what I might do wrong?
\nThanks,
\nC
\n[/quote]
", "rawMarkdown": "", "votes": null }, { "id": "61528", "postDate": "12/19/2014 08:39:46", "content": "It seems you need either Intel MKL or NVIDIA CUDA installed to use this?
\nbuild.sh blas=1 =>
\n./mshadow/mshadow/tensor_base.h:83:12: fatal error: 'cublas.h' file not found
\nMSHADOW_USE_CUDA=0 ./build.sh blas=1 =>
\n./mshadow/mshadow/tensor_base.h:282:13: error: use of undeclared identifier 'exit'
", "rawMarkdown": "", "votes": null }, { "id": "61575", "postDate": "12/20/2014 14:48:42", "content": "[quote=Michael Bewley;61471]
\nThanks Bing, I just got it working and submitted results.
\nFor others - I'm using Ubuntu 14.04, and had to muck around with the CXXNet Makefiles a bit, adding -I and -L references to the CUDA library locations.
\n[/quote]
\n\nMichael, I followed the same path you took but unfortunately, when I run cxxnet bowl.conf, I get a segmentation fault.
\nWould you care to confirm the md5sum for the 2 bin files? I get :
\na96b2dc558eb9662b417d9b23ca0fdd4 train.bin
\n891b7e1072d146f850d43cad8b0dd808 test.bin
\nThanks!
", "rawMarkdown": "", "votes": null }, { "id": "61581", "postDate": "12/20/2014 16:07:01", "content": "[quote=gregl;61575]
\n[quote=Michael Bewley;61471]
\nThanks Bing, I just got it working and submitted results.
\nFor others - I'm using Ubuntu 14.04, and had to muck around with the CXXNet Makefiles a bit, adding -I and -L references to the CUDA library locations.
\n[/quote]
\nMichael, I followed the same path you took but unfortunately, when I run cxxnet bowl.conf, I get a segmentation fault.
\nWould you care to confirm the md5sum for the 2 bin files? I get :
\na96b2dc558eb9662b417d9b23ca0fdd4 train.bin
\n891b7e1072d146f850d43cad8b0dd808 test.bin
\nThanks!
\n[/quote]
\nFor me:
\n50cd388bc640b63e2144b551fb6260fd test.bin
9f847cc39b7fee93e842a668cd4e72b4 train.bin
for me:
\nff0bee47c19b6c7ab79b29d3c37872b2
79343a11c7f80cd3b8e94712ae42cdc8
I realize the checksum of the bin file is probably irrelevant because machine dependent...
\nAnyway I fixed the issue and was able to get the 1.38 score. It seems my problem was to have built opencv before installing CUDA.
", "rawMarkdown": "", "votes": null }, { "id": "61599", "postDate": "12/20/2014 19:56:25", "content": "[quote=gregl;61598]
\nI realize the checksum of the bin file is probably irrelevant because machine dependent...
\n[/quote]
\nI don't think it is true.
\nWhat is your md5 checksums now?
", "rawMarkdown": "", "votes": null }, { "id": "61600", "postDate": "12/20/2014 20:28:29", "content": "[quote=clustifier;61599]
\n[quote=gregl;61598]
\nI realize the checksum of the bin file is probably irrelevant because machine dependent...
\n[/quote]
\nI don't think it is true.
\nWhat is your md5 checksums now?
\n[/quote]
\n\nSame as before. im2bin was working all along for me, only cxxnet was not.
", "rawMarkdown": "", "votes": null }, { "id": "61601", "postDate": "12/20/2014 21:18:16", "content": "thanks.
\nbut why our md5's are different?
", "rawMarkdown": "", "votes": null }, { "id": "61632", "postDate": "12/21/2014 19:34:01", "content": "[quote=clustifier;61601]
\nthanks.
\nbut why our md5's are different?
\n[/quote]
\n\nI built im2bin with blas, not the default mlk.
\nAlso, upstream, the md5sum of train.lst is:
\n399ab27e818cabcb5bd2158c15451a80 train.lst
\nwith the first 5 lines:
\n3406 10 data/train/chaetognath_non_sagitta/23677.jpg
22212 90 data/train/radiolarian_chain/157296.jpg
19772 83 data/train/protist_fuzzy_olive/147898.jpg
23435 98 data/train/siphonophore_calycophoran_rocketship_young/36262.jpg
18710 72 data/train/hydromedusae_solmaris/47760.jpg
My train.lst is different. I think the issue is that os.listdir does not guarantee any ordering of the results.
\n[quote=gregl;61632]
\n[quote=clustifier;61601]
\nthanks.
\nbut why our md5's are different?
\n[/quote]
\nI built im2bin with blas, not the default mlk.
\nAlso, upstream, the md5sum of train.lst is:
\n399ab27e818cabcb5bd2158c15451a80 train.lst
\nwith the first 5 lines:
\n3406 10 data/train/chaetognath_non_sagitta/23677.jpg
22212 90 data/train/radiolarian_chain/157296.jpg
19772 83 data/train/protist_fuzzy_olive/147898.jpg
23435 98 data/train/siphonophore_calycophoran_rocketship_young/36262.jpg
18710 72 data/train/hydromedusae_solmaris/47760.jpg
[/quote]
", "rawMarkdown": "", "votes": null }, { "id": "61726", "postDate": "12/23/2014 01:56:21", "content": "Include stdlib.h in tensor_base.h to fix the error about 'exit'.
\n\n\n[quote=Synthient;61528]
\nIt seems you need either Intel MKL or NVIDIA CUDA installed to use this?
\nbuild.sh blas=1 =>
\n./mshadow/mshadow/tensor_base.h:83:12: fatal error: 'cublas.h' file not found
\nMSHADOW_USE_CUDA=0 ./build.sh blas=1 =>
\n./mshadow/mshadow/tensor_base.h:282:13: error: use of undeclared identifier 'exit'
\n[/quote]
", "rawMarkdown": "", "votes": null }, { "id": "61732", "postDate": "12/23/2014 05:19:24", "content": "Hi, this is quite interesting. CXXNET was way easier to setup compared to caffe. I have some quick questions about CXXNET though:
\n- is it possible to resume training from a saved model file?
\n- is it possible to extract features from a particular layer?
\n- is there some documentation about what the various parameters in the config file mean? eg. wmat:wd
\n- how is the performance of cxxnet vs caffe?
\n\n\n[quote=Bing Xu;60426]
\nHi all,
\nI just made a guide of how to use conv net to achieve NLL of 1.382285. On a modern GPU, it only takes 5 minute to get this result.
\nPlease check https://github.com/antinucleon/cxxnet/tree/master/example/kaggle_bowl , and don't forget to star CXXNET :)
\nCXXNET is yet another neural network toolkit. It is able to run on NVIDIA GPU or CPU. If you want to run on CPU, you only need OpenCV & MKL(or OpenBlas/ATLAS) installed. No other dependence is required.
\nPS. I made it in a hurry, so if there is any problem please let me know!
\n[/quote]
", "rawMarkdown": "", "votes": null }, { "id": "61733", "postDate": "12/23/2014 05:53:49", "content": "Please refer to the wiki on Github https://github.com/antinucleon/cxxnet/wiki
\n[quote=Nissan Pow;61732]
\nHi, this is quite interesting. CXXNET was way easier to setup compared to caffe. I have some quick questions about CXXNET though:
\n- is it possible to resume training from a saved model file?
\n- is it possible to extract features from a particular layer?
\n- is there some documentation about what the various parameters in the config file mean? eg. wmat:wd
\n- how is the performance of cxxnet vs caffe?
\n[quote=Bing Xu;60426]
\nHi all,
\nI just made a guide of how to use conv net to achieve NLL of 1.382285. On a modern GPU, it only takes 5 minute to get this result.
\nPlease check https://github.com/antinucleon/cxxnet/tree/master/example/kaggle_bowl , and don't forget to star CXXNET :)
\nCXXNET is yet another neural network toolkit. It is able to run on NVIDIA GPU or CPU. If you want to run on CPU, you only need OpenCV & MKL(or OpenBlas/ATLAS) installed. No other dependence is required.
\nPS. I made it in a hurry, so if there is any problem please let me know!
\n[/quote]
\n[/quote]
", "rawMarkdown": "", "votes": null }, { "id": "61783", "postDate": "12/23/2014 22:04:51", "content": "Has someone used CXXNET on mac osx successfully?
\n(with or without cuda)
", "rawMarkdown": "", "votes": null }, { "id": "61786", "postDate": "12/23/2014 22:18:49", "content": "[quote=clustifier;61783]
\nHas someone used CXXNET on mac osx successfully?
\n(with or without cuda)
\n[/quote]
\nNope, trying though... can't seem to get by the includes. The file cblas.h is actually part of the Accelerate framework on the Mac, but now we are trying to find cublas.h. Fetched mshadow just fine...
\n====================================================================
\n$ MSHADOW_USE_CUDA=0 ./build.sh blas=1
\ng++ -c -Wall -g -O3 -msse3 -Wno-unknown-pragmas -funroll-loops -I./mshadow/ -I/usr/lib -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_CBLAS=1 -o cxxnet_data.o cxxnet/io/cxxnet_data.cpp
In file included from cxxnet/io/cxxnet_data.cpp:6:
In file included from ./mshadow/mshadow/tensor.h:11:
./mshadow/mshadow/tensor_base.h:83:12: fatal error: 'cublas.h' file not found
#include <cublas.h>
^
1 error generated.
make: *** [cxxnet_data.o] Error 1
Yes, I'm using it on 10.10 with cuda. Where are you stuck at?
\n\n[quote=clustifier;61783]
\nHas someone used CXXNET on mac osx successfully?
\n(with or without cuda)
\n[/quote]
", "rawMarkdown": "", "votes": null }, { "id": "61793", "postDate": "12/23/2014 23:24:25", "content": "cublas.h is located at /usr/local/cuda/include/, which is a symlink to /Developer/NVIDIA/CUDA-6.5/include for me.
\nAdd -I/usr/local/cuda/include/ to your Makefile.
\n\n\n[quote=Mark Conway;61786]
\n[quote=clustifier;61783]
\nHas someone used CXXNET on mac osx successfully?
\n(with or without cuda)
\n[/quote]
\nNope, trying though... can't seem to get by the includes. The file cblas.h is actually part of the Accelerate framework on the Mac, but now we are trying to find cublas.h. Fetched mshadow just fine...
\n====================================================================
\n$ MSHADOW_USE_CUDA=0 ./build.sh blas=1
\ng++ -c -Wall -g -O3 -msse3 -Wno-unknown-pragmas -funroll-loops -I./mshadow/ -I/usr/lib -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_CBLAS=1 -o cxxnet_data.o cxxnet/io/cxxnet_data.cpp
In file included from cxxnet/io/cxxnet_data.cpp:6:
In file included from ./mshadow/mshadow/tensor.h:11:
./mshadow/mshadow/tensor_base.h:83:12: fatal error: 'cublas.h' file not found
#include <cublas.h>
^
1 error generated.
make: *** [cxxnet_data.o] Error 1
[/quote]
", "rawMarkdown": "", "votes": null }, { "id": "61828", "postDate": "12/24/2014 14:51:04", "content": "Hi, anyone here uses MacOS and successfully install CXXNET? I tried to but met tons of ERRORs T.T
\n------------------------
\nFirst I use 'brew install' to get opencv and installed CUDA toolbox. Then I run ' sudo ./build.sh blas=1' and failed with 'cannot found cblas.h'. I googled cblas and found it is included in Mac's Accelerate framework. So I naively thought replacing cblas.h with Accelerate.h will solve this problem.
\n#if MSHADOW_USE_CBLAS
#include Accelerate/Accelerate.h
------------------------
\nThen I met following error:
\n/usr/include/dispatch/object.h(143): error: expected an identifier/usr/include/dispatch/object.h(362): error: identifier "dispatch_block_t" is undefined/System/Library/Frameworks/CoreFoundation.framework/Headers/CFBundle.h(148): error: format argument does not have string type
I solved this under the guide in :http://stackoverflow.com/questions/26527077/compiling-with-accelerate-framework-on-osx-yosemite
\n------------------------
\nThen this error occurs: 'ld: library not found for -lcudart' this is easy to solve, just edit the -I and -L args. on my macbook it is:
\n-I/Developer/NVIDIA/CUDA-6.5/include/ -L/Developer/NVIDIA/CUDA-6.5/lib
\n------------------------
\nThen I got this error:
\nUndefined symbols for architecture x86_64:
"std::string::_Rep::_M_destroy(std::allocator<char> const&)", referenced from:
cxxnet::SGHMCUpdater<mshadow::gpu, 1>::SGHMCUpdater(mshadow::Random<mshadow::gpu>&, mshadow::Tensor<mshadow::gpu, 1>&, mshadow::Tensor<mshadow::gpu, 1>&, char const*) in cxxnet_nnet_gpu.o
......
std::vector<cxxnet::ILayer*, std::allocator<cxxnet::ILayer*> >::_M_insert_aux(__gnu_cxx::__normal_iterator<cxxnet::ILayer**, std::vector<cxxnet::ILayer*, std::allocator<cxxnet::ILayer*> > >, cxxnet::ILayer* const&) in cxxnet_nnet_gpu.o
std::vector<cxxnet::Node<mshadow::gpu>, std::allocator<cxxnet::Node<mshadow::gpu> > >::_M_insert_aux(__gnu_cxx::__normal_iterator<cxxnet::Node<mshadow::gpu>*, std::vector<cxxnet::Node<mshadow::gpu>, std::allocator<cxxnet::Node<mshadow::gpu> > > >, cxxnet::Node<mshadow::gpu> const&) in cxxnet_nnet_gpu.o
...
ld: symbol(s) not found for architecture x86_64
I googled it and found this may due to the update of xcode from 5.x to 6.0(I use the newest xcode). And I have no idea how to fix it.
\n------------------------
\nFinally I give up using cblas and turn to mkl in https://software.intel.com/en-us/intel-parallel-studio-xe. Currently I am downloading the 30days free trial for XE (does XE have a free version for student? I could not find any link to it)
\n------------------------
\nupdate: I installed OpenBLAS and add '-I/opt/OpenBLAS/include' in makefile but still got an 'ld: symbol(s) not found for architecture x86_64' error. it seems CXXNET just cannot run on newest MacOS/xcode. Now I have give up and found alternatives.
", "rawMarkdown": "", "votes": null }, { "id": "61830", "postDate": "12/24/2014 15:15:35", "content": "I took the same route you did. I installed Intel Parallel Studio to get MKL, then set up all the include paths, and was able to at least compile some of the objects. But then the make was trying to compile with "nvcc" to create the cxxnet executable and that died a horrible death. I gave up after a few hours... it was like hacking through the jungle with a machete. Best of luck!
", "rawMarkdown": "", "votes": null }, { "id": "61848", "postDate": "12/25/2014 01:25:02", "content": "[quote=Bing Xu;60426]
\nHi all,
\nI just made a guide of how to use conv net to achieve NLL of 1.382285. On a modern GPU, it only takes 5 minute to get this result.
\nPlease check https://github.com/antinucleon/cxxnet/tree/master/example/kaggle_bowl , and don't forget to star CXXNET :)
\nCXXNET is yet another neural network toolkit. It is able to run on NVIDIA GPU or CPU. If you want to run on CPU, you only need OpenCV & MKL(or OpenBlas/ATLAS) installed. No other dependence is required.
\nPS. I made it in a hurry, so if there is any problem please let me know!
\n[/quote]
\nMay I ask about the training NLL and the distribution of predicted labels for test data?
", "rawMarkdown": "", "votes": null }, { "id": "61849", "postDate": "12/25/2014 01:29:09", "content": "[quote=Bing Xu;60426]
\nHi all,
\nI just made a guide of how to use conv net to achieve NLL of 1.382285. On a modern GPU, it only takes 5 minute to get this result.
\nPlease check https://github.com/antinucleon/cxxnet/tree/master/example/kaggle_bowl , and don't forget to star CXXNET :)
\nCXXNET is yet another neural network toolkit. It is able to run on NVIDIA GPU or CPU. If you want to run on CPU, you only need OpenCV & MKL(or OpenBlas/ATLAS) installed. No other dependence is required.
\nPS. I made it in a hurry, so if there is any problem please let me know!
\n[/quote]
\nMay I ask about the training NLL and the distribution of predicted labels for test data?
", "rawMarkdown": "", "votes": null }, { "id": "61887", "postDate": "12/26/2014 02:20:38", "content": "Hi Bing:
\nRunning on Mac Yosemite, got everything built, converted the images, but now getting the following error:
\nError:Assignment: Shape of Tensors in expression is not consistent with target
\nlocalhost:kaggle_bowl$ ../../bin/cxxnet bowl.conf
CXXNetTrainer, devCPU=1
ConvolutionLayer: nstep=256
ConvolutionLayer: nstep=256
ConvolutionLayer: nstep=128
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
node[0].shape: 256,3,48,48
node[1].shape: 256,96,12,12
node[2].shape: 256,96,12,12
node[3].shape: 256,96,6,6
node[4].shape: 256,128,8,8
node[5].shape: 256,128,8,8
node[6].shape: 256,128,8,8
node[7].shape: 256,128,8,8
node[8].shape: 256,128,4,4
node[9].shape: 1,1,256,2048
node[10].shape: 1,1,256,512
node[11].shape: 1,1,256,512
node[12].shape: 1,1,256,512
node[13].shape: 1,1,256,512
node[14].shape: 1,1,256,121
ThreadImagePageIterator:image_list=./train.lst, bin=./train.bin
cannot find models/image_mean.bin: create mean image, this will take some time...
Error:Assignment: Shape of Tensors in expression is not consistent with target
I have couple of questions with cxxnet/convert tools. First, why "convert -resize 48x48\\!" has exclamation mark? I can't find anything in convert/ImageMagik documentation. If I remove it I got similar error as Mark Conway posted, if I keep it I don't have this issue. Second question, when I run cxxnet with default configuration, it exit with the following:
\nThreadImagePageIterator:image_list=./train.lst, bin=./train.bin
cannot find models/image_mean.bin: create mean image, this will take some time...
[ 30000] images processed, 1 sec elapsedsave mean image to models/image_mean.bin..
ThreadBufferIterator: buffer_size=2
ThreadImagePageIterator:image_list=./train.lst, bin=./train.bin
loading mean image from models/image_mean.bin
ThreadBufferIterator: buffer_size=2
initializing end, start working
update round 0
and only generates models/0000.model file and models/image_mean.bin? The exit status I get (I printed via echo $?) is 1. No other messages. I don't know what's going wrong, do I have insufficient RAM or program is crashed? The code uses Quadro NVS 295 GPU.
\nThanks,
\nValentin
", "rawMarkdown": "", "votes": null }, { "id": "61919", "postDate": "12/26/2014 23:01:27", "content": "The exclamation mark means ignore aspect ratio:
\n[quote]
\nIgnore Aspect Ratio ('!' flag)
If you want you can force "-resize" to ignore the aspect ratio and distort the image so it always generates an image exactly the size specified. This is done by adding the character '!' to the size. Unfortunately this character is also sometimes used for special purposes by various UNIX command line shells. So you may have to escape the character somehow to preserve it.
[/quote]
\nReference: http://www.imagemagick.org/Usage/resize/
\n\n[quote=vk.net;61917]
\nwhy "convert -resize 48x48\\!" has exclamation mark?
\n[/quote]
", "rawMarkdown": "", "votes": null }, { "id": "61929", "postDate": "12/27/2014 13:12:12", "content": "I've written a summary of the installation of cxxnet on CentOS via blas, based on the error I've encountered during the installation.
\nHere is the summary: https://github.com/billy-inn/cxxnet/tree/master/Install
\nHope it may help.
", "rawMarkdown": "", "votes": null }, { "id": "61934", "postDate": "12/27/2014 15:26:39", "content": "Hi Valentin:
\nI'm getting pretty close. I believe the "Shape of Tensors not consistent" error is related to the image list processed by cxxnet, as you point out. So, I will experiment with the convert options and see if we can finish this last step.
\nRegards,
\nMark
\n[quote=vk.net;61917]
\nI have couple of questions with cxxnet/convert tools. First, why "convert -resize 48x48\\!" has exclamation mark? I can't find anything in convert/ImageMagik documentation. If I remove it I got similar error as Mark Conway posted, if I keep it I don't have this issue. Second question, when I run cxxnet with default configuration, it exit with the following:
\nThreadImagePageIterator:image_list=./train.lst, bin=./train.bin
cannot find models/image_mean.bin: create mean image, this will take some time...
[ 30000] images processed, 1 sec elapsedsave mean image to models/image_mean.bin..
ThreadBufferIterator: buffer_size=2
ThreadImagePageIterator:image_list=./train.lst, bin=./train.bin
loading mean image from models/image_mean.bin
ThreadBufferIterator: buffer_size=2
initializing end, start working
update round 0
and only generates models/0000.model file and models/image_mean.bin? The exit status I get (I printed via echo $?) is 1. No other messages. I don't know what's going wrong, do I have insufficient RAM or program is crashed? The code uses Quadro NVS 295 GPU.
\nThanks,
\nValentin
\n[/quote]
", "rawMarkdown": "", "votes": null }, { "id": "62072", "postDate": "12/30/2014 00:17:00", "content": "Nissan, can you elaborate where to add -I/usr/local/cuda/include in the Makefile? I'm a noob so it isn't obvious. :)
\n[quote=Nissan Pow;61793]
\ncublas.h is located at /usr/local/cuda/include/, which is a symlink to /Developer/NVIDIA/CUDA-6.5/include for me.
\nAdd -I/usr/local/cuda/include/ to your Makefile.
\n[quote=Mark Conway;61786]
\n[quote=clustifier;61783]
\nHas someone used CXXNET on mac osx successfully?
\n(with or without cuda)
\n[/quote]
\nNope, trying though... can't seem to get by the includes. The file cblas.h is actually part of the Accelerate framework on the Mac, but now we are trying to find cublas.h. Fetched mshadow just fine...
\n====================================================================
\n$ MSHADOW_USE_CUDA=0 ./build.sh blas=1
\ng++ -c -Wall -g -O3 -msse3 -Wno-unknown-pragmas -funroll-loops -I./mshadow/ -I/usr/lib -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_CBLAS=1 -o cxxnet_data.o cxxnet/io/cxxnet_data.cpp
In file included from cxxnet/io/cxxnet_data.cpp:6:
In file included from ./mshadow/mshadow/tensor.h:11:
./mshadow/mshadow/tensor_base.h:83:12: fatal error: 'cublas.h' file not found
#include <cublas.h>
^
1 error generated.
make: *** [cxxnet_data.o] Error 1
[/quote]
\n[/quote]
", "rawMarkdown": "", "votes": null }, { "id": "62073", "postDate": "12/30/2014 00:45:14", "content": "Here the part of Makefile I modified:
\nexport CFLAGS = -Wall -g -O3 -msse3 -Wno-unknown-pragmas -funroll-loops -I./mshadow/
CFLAGS+=-I/opt/cuda/include
ifeq ($(blas),1)
LDFLAGS= -L/opt/cuda/lib -lm -lcudart -lcublas -lcurand -lz `pkg-config --libs opencv` -lcblas -lgfortran -lblas
CFLAGS+= -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_CBLAS=1
else
LDFLAGS= -lm -lcudart -lcublas -lmkl_core -lmkl_intel_lp64 -lmkl_intel_thread -liomp5 -lpthread -lcurand -lz `pkg-config --libs opencv`
endif
Basically you add additional CFLAGS with you CUDA include path and modiy LDFLAGS as well with path to CUDA libraries. The CUDA toolkit (https://developer.nvidia.com/cuda-toolkit) comes from NVidia web site and it includes cublas.h
", "rawMarkdown": "", "votes": null }, { "id": "62083", "postDate": "12/30/2014 04:34:15", "content": "Thanks Bing for this eye opening example by running it with GPU Cuda cores.
\nAs much as I can follow you use one of Linux flavor to utilize GPU and maybe some other people has success on MacOS. I can run your examples on CPU on Ubuntu VM with vmplayer on a Win8.1 host however I wonder whether I can utilize host Nvidia GPU tied to host Win8.1. Is there Anybody managed to run this on Win8.1 OS with GPU? There is a MinGW solution: http://pavel.surmenok.com/2014/05/31/installing-theano-with-gpu-on-windows-64-bit/
\nbut not yet tried maybe somebody can point a better, like cygwin possibility?
\nThanks
", "rawMarkdown": "", "votes": null }, { "id": "62150", "postDate": "12/31/2014 21:30:13", "content": "Hi everyone, finally I installed cxxnet on my laptop, which has a low-end Geforce GT 630m graphic card. The score is 1.42 and the training time is one hour. But I'm still happy it works. Thank you, Bing! :D
", "rawMarkdown": "", "votes": null }, { "id": "62194", "postDate": "01/01/2015 23:43:31", "content": "I am using a CNN approach using Caffe rather than cxxnet. My network structure is very similar to what is defined in Bing's shared code. Out of interest, can anyone tell me how many training iterations the cxxnet code runs for? I can't seem to see where that is defined in the code.
", "rawMarkdown": "", "votes": null }, { "id": "62204", "postDate": "01/02/2015 07:29:46", "content": "[quote=Senecaur;62194]
\nI am using a CNN approach using Caffe rather than cxxnet. My network structure is very similar to what is defined in Bing's shared code. Out of interest, can anyone tell me how many training iterations the cxxnet code runs for? I can't seem to see where that is defined in the code.
\n[/quote]
\n\n45 in Bing's example. The entries in the .conf file are "max_round" and "num_round".
\n\nG
", "rawMarkdown": "", "votes": null }, { "id": "62214", "postDate": "01/02/2015 14:56:46", "content": "[quote=gregl;61632]
\n[quote=clustifier;61601]
\nthanks.
\nbut why our md5's are different?
\n[/quote]
\nI built im2bin with blas, not the default mlk.
\nAlso, upstream, the md5sum of train.lst is:
\n399ab27e818cabcb5bd2158c15451a80 train.lst
\nwith the first 5 lines:
\n3406 10 data/train/chaetognath_non_sagitta/23677.jpg
22212 90 data/train/radiolarian_chain/157296.jpg
19772 83 data/train/protist_fuzzy_olive/147898.jpg
23435 98 data/train/siphonophore_calycophoran_rocketship_young/36262.jpg
18710 72 data/train/hydromedusae_solmaris/47760.jpg
[/quote]
\nhow did you build im2bin with blas when mshadow needs mkl?
\n\nEDIT: Nevermind
", "rawMarkdown": "", "votes": null }, { "id": "62236", "postDate": "01/03/2015 17:39:20", "content": "Hi Valentin,
\nin case you're still having the issue you described in your second question, I had exactly the same with a GTX 680 GPU.
\nI discovered a warning during the compilation of 'tensor_gpu-inl.cuh' which is part of mshadow and located in 'cxxnet-master/mshadow/mshadow/cuda'. The warning said that the CUDA architecture of the GPU could not be determined and will be set to 2.0 automatically. It seemed that the __CUDA_ARCH__ macro, that is checked for the architecture, was broken. Setting this macro by hand like
\n#define __CUDA_ARCH__ 300
\nresolved the issue for me (in my case, the 300 comes from the GTX 680 having compute capability 3.0). Maybe this can help with your problem too...
\nBest,
\nMatthias
\np.s.: Thanks a lot Bing for making this available!
\n[quote=vk.net;61917]
\nSecond question, when I run cxxnet with default configuration, it exit with the following:
\nThreadImagePageIterator:image_list=./train.lst, bin=./train.bin
cannot find models/image_mean.bin: create mean image, this will take some time...
[ 30000] images processed, 1 sec elapsedsave mean image to models/image_mean.bin..
ThreadBufferIterator: buffer_size=2
ThreadImagePageIterator:image_list=./train.lst, bin=./train.bin
loading mean image from models/image_mean.bin
ThreadBufferIterator: buffer_size=2
initializing end, start working
update round 0
and only generates models/0000.model file and models/image_mean.bin? The exit status I get (I printed via echo $?) is 1. No other messages. I don't know what's going wrong, do I have insufficient RAM or program is crashed? The code uses Quadro NVS 295 GPU.
\nThanks,
\nValentin
\n[/quote]
", "rawMarkdown": "", "votes": null }, { "id": "62285", "postDate": "01/05/2015 03:08:59", "content": "[quote=vk.net;62073]
\nHere the part of Makefile I modified:
\nexport CFLAGS = -Wall -g -O3 -msse3 -Wno-unknown-pragmas -funroll-loops -I./mshadow/
CFLAGS+=-I/opt/cuda/include
ifeq ($(blas),1)
LDFLAGS= -L/opt/cuda/lib -lm -lcudart -lcublas -lcurand -lz `pkg-config --libs opencv` -lcblas -lgfortran -lblas
CFLAGS+= -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_CBLAS=1
else
LDFLAGS= -lm -lcudart -lcublas -lmkl_core -lmkl_intel_lp64 -lmkl_intel_thread -liomp5 -lpthread -lcurand -lz `pkg-config --libs opencv`
endif
Basically you add additional CFLAGS with you CUDA include path and modiy LDFLAGS as well with path to CUDA libraries. The CUDA toolkit (https://developer.nvidia.com/cuda-toolkit) comes from NVidia web site and it includes cublas.h
\n[/quote]
\n\nThanks for the advice; this resolved the cublas issue. I was wondering if it worked for you with just that, though; after making the modification, I now get
\nsh build.sh blas=1
\n
Fetch mshadow...
fatal: destination path 'mshadow' already exists and is not an empty directory.
nvcc -c -o cxxnet_nnet_gpu.o --use_fast_math -g -O3 -ccbin g++ -Xcompiler "-Wall -g -O3 -msse3 -Wno-unknown-pragmas -funroll-loops -I./mshadow/ -I/usr/local/cuda/include -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_CBLAS=1" cxxnet/nnet/cxxnet_nnet.cu
/bin/sh: 1: nvcc: not found
make: *** [cxxnet_nnet_gpu.o] Error 127
Torgos, looks like you havn't added CUDA to your path?? i.e. something like:
\nexport PATH=$PATH:/usr/local/cuda-6.5/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-6.5/lib64:/usr/local/cuda-6.5/include
Right you are; I thought I had added it to my bashrc. Thanks!
\nEdit.Unfortunately still seems to be broken, though. Blargh. Has anyone gotten this installed on a Ubuntu 14.04 on an Amazon GPU instance?
\nEdit 2: I think I got it; the ubuntu distribution is 64 bit, so you will get compatibility issues with the 32 bit library; point the makefile at -L/usr/local/cuda/lib64 instead. You also need to add the -pthread to
\nCFLAGS+= -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_CBLAS=1 -pthread
\nas per
\nhttps://github.com/antinucleon/cxxnet/issues/14
\nNo errors (other than 'mshadow already exists') when running sh build.sh blas=1 after that. Still need to figure out what paths to change the Makefile for im2bin, though.
\nEdit 3. Modify the first few lines in the im2bin Makefile to
\nexport CFLAGS = -Wall -O3 -msse3 -Wno-unknown-pragmas -funroll-loops -I../mshadow/ -I..
\nCFLAGS+=-I/usr/local/cuda/include
export LDFLAGS=-L/usr/local/cuda/lib64 -lm -lcudart -lcublas -lcurand -lz `pkg-config --libs opencv` -lblas
export NVCCFLAGS = -g -O3 -ccbin $(CXX)
CFLAGS+= -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_CBLAS=1 -pthread
It should compile correctly now; unfortunately, the actual command doesn't work! I suspect that the issue was in the argument passed when train.lst was generated.
", "rawMarkdown": "", "votes": null }, { "id": "62324", "postDate": "01/06/2015 02:34:55", "content": "Did anyone get it running on the Macbook on OSX?
", "rawMarkdown": "", "votes": null }, { "id": "62328", "postDate": "01/06/2015 04:55:00", "content": "I stumbled across this on the caffe github repo today:
\n"Be aware of the competing fast GPU neural network library CXXNET"
\nhttps://github.com/BVLC/caffe/issues/382
\nI am so impressed with with what Tianqi and Bing have accomplished with cxxnet and xgboost. Please, keep doing what you're doing!
", "rawMarkdown": "", "votes": null }, { "id": "62332", "postDate": "01/06/2015 06:50:55", "content": "Okay, I got past a few problems but now I'm running into this:
\nUndefined symbols for architecture x86_64:
"std::string::_Rep::_M_destroy(std::allocator<char> const&)", referenced from:
cxxnet::SGHMCUpdater<mshadow::gpu, 1>::SGHMCUpdater(mshadow::Random<mshadow::gpu>&, mshadow::Tensor<mshadow::gpu, 1>&, mshadow::Tensor<mshadow::gpu, 1>&, char const*) in cxxnet_nnet_gpu.o
cxxnet::SGHMCUpdater<mshadow::gpu, 1>::~SGHMCUpdater() in cxxnet_nnet_gpu.o
cxxnet::SGDUpdater<mshadow::gpu, 1>::SGDUpdater(mshadow::Tensor<mshadow::gpu, 1>&, mshadow::Tensor<mshadow::gpu, 1>&, char const*) in cxxnet_nnet_gpu.o
cxxnet::SGDUpdater<mshadow::gpu, 1>::~SGDUpdater() in cxxnet_nnet_gpu.o
cxxnet::SGDUpdater<mshadow::gpu, 1>::~SGDUpdater() in cxxnet_nnet_gpu.o
cxxnet::SGHMCUpdater<mshadow::gpu, 3>::SGHMCUpdater(mshadow::Random<mshadow::gpu>&, mshadow::Tensor<mshadow::gpu, 3>&, mshadow::Tensor<mshadow::gpu, 3>&, char const*) in cxxnet_nnet_gpu.o
cxxnet::SGHMCUpdater<mshadow::gpu, 3>::~SGHMCUpdater() in cxxnet_nnet_gpu.o
...
"std::string::_Rep::_S_empty_rep_storage", referenced from:
cxxnet::SGHMCUpdater<mshadow::gpu, 1>::SGHMCUpdater(mshadow::Random<mshadow::gpu>&, mshadow::Tensor<mshadow::gpu, 1>&, mshadow::Tensor<mshadow::gpu, 1>&, char const*) in cxxnet_nnet_gpu.o
cxxnet::SGHMCUpdater<mshadow::gpu, 1>::~SGHMCUpdater() in cxxnet_nnet_gpu.o
cxxnet::SGDUpdater<mshadow::gpu, 1>::SGDUpdater(mshadow::Tensor<mshadow::gpu, 1>&, mshadow::Tensor<mshadow::gpu, 1>&, char const*) in cxxnet_nnet_gpu.o
cxxnet::SGDUpdater<mshadow::gpu, 1>::~SGDUpdater() in cxxnet_nnet_gpu.o
cxxnet::SGDUpdater<mshadow::gpu, 1>::~SGDUpdater() in cxxnet_nnet_gpu.o
cxxnet::SGHMCUpdater<mshadow::gpu, 3>::SGHMCUpdater(mshadow::Random<mshadow::gpu>&, mshadow::Tensor<mshadow::gpu, 3>&, mshadow::Tensor<mshadow::gpu, 3>&, char const*) in cxxnet_nnet_gpu.o
cxxnet::SGHMCUpdater<mshadow::gpu, 3>::~SGHMCUpdater() in cxxnet_nnet_gpu.o
...
"std::string::assign(char const*, unsigned long)", referenced from:
cxxnet::SGHMCUpdater<mshadow::gpu, 1>::SGHMCUpdater(mshadow::Random<mshadow::gpu>&, mshadow::Tensor<mshadow::gpu, 1>&, mshadow::Tensor<mshadow::gpu, 1>&, char const*) in cxxnet_nnet_gpu.o
cxxnet::SGDUpdater<mshadow::gpu, 1>::SGDUpdater(mshadow::Tensor<mshadow::gpu, 1>&, mshadow::Tensor<mshadow::gpu, 1>&, char const*) in cxxnet_nnet_gpu.o
cxxnet::SGHMCUpdater<mshadow::gpu, 3>::SGHMCUpdater(mshadow::Random<mshadow::gpu>&, mshadow::Tensor<mshadow::gpu, 3>&, mshadow::Tensor<mshadow::gpu, 3>&, char const*) in cxxnet_nnet_gpu.o
cxxnet::SGDUpdater<mshadow::gpu, 3>::SGDUpdater(mshadow::Tensor<mshadow::gpu, 3>&, mshadow::Tensor<mshadow::gpu, 3>&, char const*) in cxxnet_nnet_gpu.o
cxxnet::SGHMCUpdater<mshadow::gpu, 2>::SGHMCUpdater(mshadow::Random<mshadow::gpu>&, mshadow::Tensor<mshadow::gpu, 2>&, mshadow::Tensor<mshadow::gpu, 2>&, char const*) in cxxnet_nnet_gpu.o
cxxnet::SGDUpdater<mshadow::gpu, 2>::SGDUpdater(mshadow::Tensor<mshadow::gpu, 2>&, mshadow::Tensor<mshadow::gpu, 2>&, char const*) in cxxnet_nnet_gpu.o
cxxnet::utils::MetricSet::AddMetric(char const*) in cxxnet_nnet_gpu.o
...
"std::string::assign(std::string const&)", referenced from:
std::vector<std::pair<std::string, std::string>, std::allocator<std::pair<std::string, std::string> > >::_M_insert_aux(__gnu_cxx::__normal_iterator<std::pair<std::string, std::string>*, std::vector<std::pair<std::string, std::string>, std::allocator<std::pair<std::string, std::string> > > >, std::pair<std::string, std::string> const&) in cxxnet_nnet_gpu.o
"std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, std::allocator<char> const&)", referenced from:
cxxnet::NetConfigHelper::SetParam(char const*, char const*) in cxxnet_nnet_gpu.o
"std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(std::string const&)", referenced from:
cxxnet::NetConfigHelper::SetParam(char const*, char const*) in cxxnet_nnet_gpu.o
std::vector<std::pair<std::string, std::string>, std::allocator<std::pair<std::string, std::string> > >::_M_insert_aux(__gnu_cxx::__normal_iterator<std::pair<std::string, std::string>*, std::vector<std::pair<std::string, std::string>, std::allocator<std::pair<std::string, std::string> > > >, std::pair<std::string, std::string> const&) in cxxnet_nnet_gpu.o
"std::__throw_length_error(char const*)", referenced from:
std::vector<float, std::allocator<float> >::_M_fill_insert(__gnu_cxx::__normal_iterator<float*, std::vector<float, std::allocator<float> > >, unsigned long, float const&) in cxxnet_nnet_gpu.o
std::vector<std::vector<float, std::allocator<float> >, std::allocator<std::vector<float, std::allocator<float> > > >::_M_fill_insert(__gnu_cxx::__normal_iterator<std::vector<float, std::allocator<float> >*, std::vector<std::vector<float, std::allocator<float> >, std::allocator<std::vector<float, std::allocator<float> > > > >, unsigned long, std::vector<float, std::allocator<float> > const&) in cxxnet_nnet_gpu.o
std::vector<float, std::allocator<float> >::_M_insert_aux(__gnu_cxx::__normal_iterator<float*, std::vector<float, std::allocator<float> > >, float const&) in cxxnet_nnet_gpu.o
std::vector<int, std::allocator<int> >::_M_fill_insert(__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int> > >, unsigned long, int const&) in cxxnet_nnet_gpu.o
std::vector<cxxnet::IUpdater*, std::allocator<cxxnet::IUpdater*> >::_M_insert_aux(__gnu_cxx::__normal_iterator<cxxnet::IUpdater**, std::vector<cxxnet::IUpdater*, std::allocator<cxxnet::IUpdater*> > >, cxxnet::IUpdater* const&) in cxxnet_nnet_gpu.o
std::vector<cxxnet::ILayer*, std::allocator<cxxnet::ILayer*> >::_M_insert_aux(__gnu_cxx::__normal_iterator<cxxnet::ILayer**, std::vector<cxxnet::ILayer*, std::allocator<cxxnet::ILayer*> > >, cxxnet::ILayer* const&) in cxxnet_nnet_gpu.o
std::vector<cxxnet::Node<mshadow::gpu>, std::allocator<cxxnet::Node<mshadow::gpu> > >::_M_insert_aux(__gnu_cxx::__normal_iterator<cxxnet::Node<mshadow::gpu>*, std::vector<cxxnet::Node<mshadow::gpu>, std::allocator<cxxnet::Node<mshadow::gpu> > > >, cxxnet::Node<mshadow::gpu> const&) in cxxnet_nnet_gpu.o
...
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [bin/cxxnet] Error 1
[quote=Torgos;62302]
\nEdit 3. Modify the first few lines in the im2bin Makefile to
\nexport CFLAGS = -Wall -O3 -msse3 -Wno-unknown-pragmas -funroll-loops -I../mshadow/ -I..
\nCFLAGS+=-I/usr/local/cuda/include
export LDFLAGS=-L/usr/local/cuda/lib64 -lm -lcudart -lcublas -lcurand -lz `pkg-config --libs opencv` -lblas
export NVCCFLAGS = -g -O3 -ccbin $(CXX)
CFLAGS+= -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_CBLAS=1 -pthread
It should compile correctly now; unfortunately, the actual command doesn't work! I suspect that the issue was in the argument passed when train.lst was generated.
\n[/quote]
This bit worked for me. Thanks.
At first the command didn't work for me either and I was wondering why. Turns out there was a permissions issue cos I compiled the im2bin on an mounted NTFS partition instead of EXT. Ended up moving the executable to the EXT partition, gave chmod +x and command ran smoothly.
To Matthias: indeed you found a remedy. I applied macro definition and recompiled the code. Now it runs!!! So, for those who experience silent crash, just follow Matthias advice and define __CUDA_ARCH__ in cxxnet/mshadow/mshadow/cuda/tensor_gpu-inl.cuh file.
", "rawMarkdown": "", "votes": null }, { "id": "62347", "postDate": "01/06/2015 13:00:13", "content": "is it possible to compile CXXNET without OpenCV?
", "rawMarkdown": "", "votes": null }, { "id": "62378", "postDate": "01/06/2015 20:28:32", "content": "I'm trying to get this running on a Macbook. What version of OpenCV is everyone using? I have version 2.4.9 and I'm getting this error:
\n\nUndefined symbols for architecture x86_64:
"cv::imread(std::string const&, int)", referenced from:
cxxnet::ImageIterator::LoadImage(mshadow::TensorContainer<mshadow::cpu, 3>&, cxxnet::DataInst&, char const*) in cxxnet_data.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [bin/cxxnet] Error 1
Any ideas?
", "rawMarkdown": "", "votes": null }, { "id": "62410", "postDate": "01/07/2015 17:22:19", "content": "Hi, when I run the code, it produces the output with train-error values for each round. Can someone explain how they're defined.
", "rawMarkdown": "", "votes": null }, { "id": "62509", "postDate": "01/09/2015 16:23:15", "content": "Hello,
\nI finally got the code compiled and linked, although a cpu version, because I have no GPU at the moment. I got the training done and found the 45 models in the models directory as expected along with the image means.
\n$ ../../bin/cxxnet bowl_48x48.conf
CXXNetTrainer, devCPU=1
ConvolutionLayer: nstep=256
.
[output removed]
\n.
\nThreadImagePageIterator:image_list=./train_48x48.lst, bin=./train_48x48.bin
cannot find models/image_mean_48x48.bin: create mean image, this will take some time...
[ 30000] images processed, 1 sec elapsedsave mean image to models/image_mean_48x48.bin..
ThreadBufferIterator: buffer_size=2
ThreadImagePageIterator:image_list=./train_48x48.lst, bin=./train_48x48.bin
loading mean image from models/image_mean_48x48.bin
ThreadBufferIterator: buffer_size=2
initializing end, start working
round 0:[ 100] 246 sec elapsed[1] train-error:0.797901
.
[output removed]
\n.
round 44:[ 100] 20950 sec elapsed[45] train-error:0.352688
updating end, 21135 sec in all
\nNext, I invoked the predictor. It continues for a while and then halts with an error Error:decoding fail as seen in the trace below. I have the output txt file, where I see 67400 images were processed. Inspecting the code, I found that error message in the file cxxnet_iter_thread_imbin-inl.hpp, line 65, function inline static void LoadImage().
\n$ ../../bin/cxxnet pred_48x48.conf
CXXNetTrainer, devCPU=1
ConvolutionLayer: nstep=100
ConvolutionLayer: nstep=100
ConvolutionLayer: nstep=100
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
node[0].shape: 100,3,48,48
node[1].shape: 100,96,12,12
node[2].shape: 100,96,12,12
node[3].shape: 100,96,6,6
node[4].shape: 100,128,8,8
node[5].shape: 100,128,8,8
node[6].shape: 100,128,8,8
node[7].shape: 100,128,8,8
node[8].shape: 100,128,4,4
node[9].shape: 1,1,100,2048
node[10].shape: 1,1,100,512
node[11].shape: 1,1,100,512
node[12].shape: 1,1,100,512
node[13].shape: 1,1,100,512
node[14].shape: 1,1,100,121
ThreadImagePageIterator:image_list=./train_48x48.lst, bin=./train_48x48.bin
loading mean image from models/image_mean_48x48.bin
ThreadBufferIterator: buffer_size=2
ThreadImagePageIterator:image_list=./test_48x48.lst, bin=./test_48x48.bin
loading mean image from models/image_mean_48x48.bin
ThreadBufferIterator: buffer_size=2
initializing end, start working
start predicting...
Error:decoding fail
I am suspecting something went wrong during the conversion and subsequently during loading of the images. Did anyone come across this situation? Any hints?
\nThanks.
\n\nEdit: Tried out again. It works.
", "rawMarkdown": "", "votes": null }, { "id": "62815", "postDate": "01/17/2015 06:15:31", "content": "Hi Mark, Can you give a brief description on how you managed to build on mac? Thank you.
\n[quote=Mark Conway;61887]
\nHi Bing:
\nRunning on Mac Yosemite, got everything built, converted the images, but now getting the following error:
\nError:Assignment: Shape of Tensors in expression is not consistent with target
\nlocalhost:kaggle_bowl$ ../../bin/cxxnet bowl.conf
CXXNetTrainer, devCPU=1
ConvolutionLayer: nstep=256
ConvolutionLayer: nstep=256
ConvolutionLayer: nstep=128
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
node[0].shape: 256,3,48,48
node[1].shape: 256,96,12,12
node[2].shape: 256,96,12,12
node[3].shape: 256,96,6,6
node[4].shape: 256,128,8,8
node[5].shape: 256,128,8,8
node[6].shape: 256,128,8,8
node[7].shape: 256,128,8,8
node[8].shape: 256,128,4,4
node[9].shape: 1,1,256,2048
node[10].shape: 1,1,256,512
node[11].shape: 1,1,256,512
node[12].shape: 1,1,256,512
node[13].shape: 1,1,256,512
node[14].shape: 1,1,256,121
ThreadImagePageIterator:image_list=./train.lst, bin=./train.bin
cannot find models/image_mean.bin: create mean image, this will take some time...
Error:Assignment: Shape of Tensors in expression is not consistent with target
[/quote]
", "rawMarkdown": "", "votes": null }, { "id": "62917", "postDate": "01/19/2015 23:03:27", "content": "Just in case it's of help to anyone is still having the below opencv x86_64 error on Mac with the latest XCode, I managed to get it compiling on mine by re-compiling opencv to use -stdlib=libstdc++ instead of the default -stdlib=libc++, via the shortcut of re-installing it with the following command:
\n"brew reinstall opencv --with-cuda"
\n(if you do a "brew edit opencv" you'll notice it's turning on -stdlib=libstdc++ when the cuda option is set). you of course need to have the cuda stuff installed as well
\nI built cxxnet with blas=1 and had these settings in the Makefile (though perhaps not all are needed, since I was messing around with it for a while)
\nLDFLAGS= -lm -lcudart -lcublas -lcurand -lz `pkg-config --libs opencv` `pkg-config --libs openblas` -lblas -L/usr/local/cuda/lib
CFLAGS+= -stdlib=libstdc++ -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_CBLAS=1 -DMSHADOW_USE_CUDA=1 -I/usr/local/Cellar/openblas/0.2.13/include -I/usr/local/cuda/include
-----
\nUndefined symbols for architecture x86_64:
"cv::imread(std::string const&, int)", referenced from:
-----
", "rawMarkdown": "", "votes": null }, { "id": "62936", "postDate": "01/20/2015 19:25:41", "content": "Wow. Thank you so much Ash. I've been blocked by this error for quite some time.
", "rawMarkdown": "", "votes": null }, { "id": "63180", "postDate": "01/26/2015 08:02:06", "content": "Hi Bing,
\nThank you for sharing the guide.
\n
In the config file you set image shape to (3,48,48). As I can understand "3" is a number of input channels. But images in dataset is in gray-scale and have only one channel. How you convert from 1 to 3 channels? Is it just a copy?
I use opencv to decode, opencv's output for jpg is 3 channel, and all are same.
\n[quote=Oleksandr Baiev;63180]
\nHi Bing,
\nThank you for sharing the guide.
\n
In the config file you set image shape to (3,48,48). As I can understand "3" is a number of input channels. But images in dataset is in gray-scale and have only one channel. How you convert from 1 to 3 channels? Is it just a copy?
[/quote]
", "rawMarkdown": "", "votes": null }, { "id": "63208", "postDate": "01/26/2015 20:40:41", "content": "Oh, thank you!
\n[quote=Bing Xu;63204]
\nI use opencv to decode, opencv's output for jpg is 3 channel, and all are same.
\n[quote=Oleksandr Baiev;63180]
\nHi Bing,
\nThank you for sharing the guide.
\n
In the config file you set image shape to (3,48,48). As I can understand "3" is a number of input channels. But images in dataset is in gray-scale and have only one channel. How you convert from 1 to 3 channels? Is it just a copy?
[/quote]
\n[/quote]
", "rawMarkdown": "", "votes": null }, { "id": "63220", "postDate": "01/26/2015 23:53:40", "content": "[quote=Bing Xu;63204]
\nI use opencv to decode, opencv's output for jpg is 3 channel, and all are same.
\n[/quote]
\nI believe libjpeg, which is required for openCV, will decode greyscale images into a single channel. I'm not sure if any additional overhead is introduced by processing three identical channels in the convnet.
\nCaffe appears to determine image size by reading the jpeg metadata and it understands greyscale. However I have found cxxnet to be substantially faster than caffe on this problem.
", "rawMarkdown": "", "votes": null }, { "id": "63272", "postDate": "01/28/2015 12:53:48", "content": "[quote=Ertuka;62083]
\nThanks Bing for this eye opening example by running it with GPU Cuda cores.
\nAs much as I can follow you use one of Linux flavor to utilize GPU and maybe some other people has success on MacOS. I can run your examples on CPU on Ubuntu VM with vmplayer on a Win8.1 host however I wonder whether I can utilize host Nvidia GPU tied to host Win8.1. Is there Anybody managed to run this on Win8.1 OS with GPU? There is a MinGW solution: http://pavel.surmenok.com/2014/05/31/installing-theano-with-gpu-on-windows-64-bit/
\nbut not yet tried maybe somebody can point a better, like cygwin possibility?
\nThanks
\n[/quote]
\nErtuka, Could you let me know the changes you made to the build, I am running on ubuntu using VMware and I am stuck with the build issues.
", "rawMarkdown": "", "votes": null }, { "id": "63287", "postDate": "01/29/2015 01:22:34", "content": "I used make file under directory make under top dir:
\ncd make; make -f Cblas.cpu.mk
\nYou need add some stuff in it, not remember exactly what but here is the lines from my version
\nthat you can see what is missing:
\nexport LDFLAGS= -lcblas -lm -lz -lpthread `pkg-config --libs opencv`
\nCFLAGS += -DMSHADOW_USE_CBLAS=1 -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_CUDA=0
\nIt creates the bin in the same directory.
\nI assume you install Cuda developemnt kit by ignoring nvidia driver otherwise next time login your VM will be in a bad state. CPU takes close to 6hours to run all 45 iterations on i7-4710HQ and not uses more than 4GB memory and a single CPU.
\nMaybe there is way to make multthreading but I havent looked at it much.
\nIn terms of running on windows, my attempt for cxxnet was a failure since it uses heavily GNU specific stuff and combined with my weak Visual studio community experience so I couldn't manage to combine nvcc on Visual studio with gnu c++ on mingw nor VC++. On windows my only success was to be able run theano on python and GTX 860M runs a simple large matrix multiplication 24 times faster than on CPU. So I am looking to dual boot my laptop to use GPU on a linux flavor after many years of satisfied user of linux on VMs.
", "rawMarkdown": "", "votes": null }, { "id": "63339", "postDate": "01/30/2015 02:49:24", "content": "Quick question, Bing; is there an easy way to specify to hold out and report error rates on a validation set during training in the conf file?
\n\nThanks!
", "rawMarkdown": "", "votes": null }, { "id": "63357", "postDate": "01/30/2015 13:31:21", "content": "[quote=kartal;61442]
\nHi Bing,
\nWhen I run the gen_train.py function I am getting these warnings "convert is not recognized as an internal or external command". Are you using a special python package to resize images?
\n[/quote]
\nInstall Imagemagick
", "rawMarkdown": "", "votes": null }, { "id": "63366", "postDate": "01/30/2015 14:56:59", "content": "Hi
\nI got CXXNET installed and it runs on the cpu. But on the gpu (Iuse dev = gpu) after initializing I get:
\ninitializing end, start working
update round 0
and it stops and jumps back to the shell promt. No error code, nothing. And no training.
\nI'm using Ubuntu 14.04 and the NVIDIA Geforce GTX 750 Ti.
\nCUDA seems to work and I even get:
\nUse CUDA Device 0: GeForce GTX 750 Ti
CXXNetTrainer, devCPU=0
after starting cxxnet.
\nAny idea what's the problem?
", "rawMarkdown": "", "votes": null }, { "id": "63368", "postDate": "01/30/2015 17:04:09", "content": "I'm working off a ubuntu 14.04 amazon AWS gpu instance (grid K520), and one of the things I had done prior to trying to work with cxxnet was go through http://tleyden.github.io/blog/2014/10/25/cuda-6-dot-5-on-aws-gpu-instance-running-ubuntu-14-dot-04/.
\nI hadn't tried to use cxxnet before doing the installation, though.
", "rawMarkdown": "", "votes": null }, { "id": "63370", "postDate": "01/30/2015 18:30:59", "content": "[quote=Wolf Gang;63366]
\nHi
\nI got CXXNET installed and it runs on the cpu. But on the gpu (Iuse dev = gpu) after initializing I get:
\ninitializing end, start working
update round 0
and it stops and jumps back to the shell promt. No error code, nothing. And no training.
\nI'm using Ubuntu 14.04 and the NVIDIA Geforce GTX 750 Ti.
\nCUDA seems to work and I even get:
\nUse CUDA Device 0: GeForce GTX 750 Ti
CXXNetTrainer, devCPU=0
after starting cxxnet.
\nAny idea what's the problem?
\n[/quote]
\nCheck this post by lametta. Doing that fixed the same issue for me https://www.kaggle.com/c/datasciencebowl/forums/t/11279/public-start-guide-of-deep-network-with-a-score-of-1-382285/62236#post62236
", "rawMarkdown": "", "votes": null }, { "id": "63426", "postDate": "02/02/2015 03:43:40", "content": "[quote=Ertuka;63287]
\nI used make file under directory make under top dir:
\ncd make; make -f Cblas.cpu.mk
\nYou need add some stuff in it, not remember exactly what but here is the lines from my version
\nthat you can see what is missing:
\nexport LDFLAGS= -lcblas -lm -lz -lpthread `pkg-config --libs opencv`
\nCFLAGS += -DMSHADOW_USE_CBLAS=1 -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_CUDA=0
\nIt creates the bin in the same directory.
\nI assume you install Cuda developemnt kit by ignoring nvidia driver otherwise next time login your VM will be in a bad state. CPU takes close to 6hours to run all 45 iterations on i7-4710HQ and not uses more than 4GB memory and a single CPU.
\nMaybe there is way to make multthreading but I havent looked at it much.
\nIn terms of running on windows, my attempt for cxxnet was a failure since it uses heavily GNU specific stuff and combined with my weak Visual studio community experience so I couldn't manage to combine nvcc on Visual studio with gnu c++ on mingw nor VC++. On windows my only success was to be able run theano on python and GTX 860M runs a simple large matrix multiplication 24 times faster than on CPU. So I am looking to dual boot my laptop to use GPU on a linux flavor after many years of satisfied user of linux on VMs.
\n[/quote]
\nThank you very much, it helped!
", "rawMarkdown": "", "votes": null }, { "id": "63438", "postDate": "02/02/2015 15:19:43", "content": "[quote=Bing Xu;63204]
\nI use opencv to decode, opencv's output for jpg is 3 channel, and all are same.
\n[/quote]
\nHas anybody had any luck getting this to run with 1 channel? I've tried to reduce my channels to 1, and my image manipulation apps all say the image is greyscale, but no matter what CXXNet isn't working for me unless I set the channel dimension to 3.
\nI have this feeling like my image apps are lying to me, and that my JPGs really are still 3 channels. Not sure what to use to legitimately make a one channel image though.
", "rawMarkdown": "", "votes": null }, { "id": "63440", "postDate": "02/02/2015 15:46:07", "content": "They are 1 channel, per Bing's note cxxnet converts them to 3 channel no matter what.
\nGetting cxxnet to handle single channel input without replicating the data three times would be a good enhancement.
", "rawMarkdown": "", "votes": null }, { "id": "63491", "postDate": "02/03/2015 18:20:35", "content": "I haven't been following this thread, but thought it might be a good way to learn something about cxxnet to see how it compares with similar tools.
\nDidn't have much trouble building it (on Linux Mint 17), although I did have to monkey with the make file and the source code slightly (nothing that hasn't been covered in this forum already). Once it was built I managed to run the MNIST example/tutorial successfully (with and without GPU acceleration). Very nice! Compares very favorably to other CNN implementations as far as speed is concerned!
\nFirst I have a simple question: The MNIST config file has "pad = 1" for the first conv layer, but the cxxnet "Layers" documentation doesn't mention a "pad" option. What is it for?
\nAnd a comment/suggestion: It would be nice to have the ability to specify a hold-out or validation set in the config file (that either isn't part of the training data, or is a random N% of the training data) against which the model is tested after each training epoch, using the selected metric, rather than (or in addition to) evaluation vs. the training data. Would help with cross-validation and finding "early stopping" criteria.
", "rawMarkdown": "", "votes": null }, { "id": "63492", "postDate": "02/03/2015 19:04:55", "content": "Never mind my padding question above. I looked at the code and it appears to mean exactly the same thing as it does in Caffe. That was my assumption, but lacking documentation I wasn't sure.
", "rawMarkdown": "", "votes": null }, { "id": "63619", "postDate": "02/04/2015 22:33:46", "content": "If we're adding feature requests, I would note that it seems strange to have both a training config file and prediction config file,given the substantial overlap in parameters between the two.
", "rawMarkdown": "", "votes": null }, { "id": "63632", "postDate": "02/05/2015 07:46:13", "content": "", "rawMarkdown": "", "votes": null }, { "id": "63669", "postDate": "02/06/2015 00:49:09", "content": "You can decode with `CV_LOAD_IMAGE_GRAYSCALE` or `CV_LOAD_IMAGE_UNCHANGED` parameter, that will give you a single channel image :)
\n[quote=Bing Xu;63204]
\nI use opencv to decode, opencv's output for jpg is 3 channel, and all are same.
\n[quote=Oleksandr Baiev;63180]
\nHi Bing,
\nThank you for sharing the guide.
\n
In the config file you set image shape to (3,48,48). As I can understand "3" is a number of input channels. But images in dataset is in gray-scale and have only one channel. How you convert from 1 to 3 channels? Is it just a copy?
[/quote]
\n[/quote]
", "rawMarkdown": "", "votes": null }, { "id": "63758", "postDate": "02/07/2015 22:16:35", "content": "After add path to header and lib files , I run "sh build.sh" and get a error message --
\n#error -- unsupported GNU version! gcc 4.9 and up are not supported!
\nIs there an easy way to get round this error?
\nThanks.
", "rawMarkdown": "", "votes": null }, { "id": "63760", "postDate": "02/07/2015 22:58:24", "content": "[quote=superfan123;63758]
\nAfter add path to header and lib files , I run "sh build.sh" and get a error message --
\n#error -- unsupported GNU version! gcc 4.9 and up are not supported!
\nIs there an easy way to get round this error?
\nThanks.
\n[/quote]
\nThe root cause is that CUDA 6.5 doesn't support gcc 4.9 or up. gcc 4.8 should be good.
", "rawMarkdown": "", "votes": null }, { "id": "63761", "postDate": "02/07/2015 23:13:11", "content": "Just modify CUDA config.h file to accept 4.9 since CUDA 6.5 contraints gcc to 4.8 and lower.
\nI have used gcc 4.9 on F21 with CUDA 6.5 and not seen any issue so far.
\n[quote=superfan123;63758]
\nAfter add path to header and lib files , I run "sh build.sh" and get a error message --
\n#error -- unsupported GNU version! gcc 4.9 and up are not supported!
\nIs there an easy way to get round this error?
\nThanks.
\n[/quote]
", "rawMarkdown": "", "votes": null }, { "id": "63762", "postDate": "02/07/2015 23:15:14", "content": "[quote=Guocong Song;63760]
\n[quote=superfan123;63758]
\nAfter add path to header and lib files , I run "sh build.sh" and get a error message --
\n#error -- unsupported GNU version! gcc 4.9 and up are not supported!
\nIs there an easy way to get round this error?
\nThanks.
\n[/quote]
\nThe root cause is that CUDA 6.5 doesn't support gcc 4.9 or up. gcc 4.8 should be good.
\n[/quote]
\nYour are right. I changed gcc link file in ~/CUDA-6.5/bin and make it point to gcc++-4.8 but that didn't fix the problem,
\nAnything else I should change?
\nI have gcc4.9 and 4.8 installed on Ubuntu 14.04.
\nThanks a lot.
", "rawMarkdown": "", "votes": null }, { "id": "63764", "postDate": "02/07/2015 23:26:59", "content": "[quote=Ertuka;63761]
\nJust modify CUDA config.h file to accept 4.9 since CUDA 6.5 contraints gcc to 4.8 and lower.
\nI have used gcc 4.9 on F21 with CUDA 6.5 and not seen any issue so far.
\n[quote=superfan123;63758]
\nAfter add path to header and lib files , I run "sh build.sh" and get a error message --
\n#error -- unsupported GNU version! gcc 4.9 and up are not supported!
\nIs there an easy way to get round this error?
\nThanks.
\n[/quote]
\n[/quote]
\nThanks Ertuka, Could you show me a sample statement to modify config.h?
", "rawMarkdown": "", "votes": null }, { "id": "63765", "postDate": "02/07/2015 23:34:03", "content": "Thanks guys, Now it's working :)
", "rawMarkdown": "", "votes": null }, { "id": "63766", "postDate": "02/08/2015 00:31:23", "content": "Now after add those header and lib --
\n-I/usr/local/cuda-6.5/include
\n-I/opt/intel/mkl/include
\n-I/opt/intel/composer_xe_2015.2.164/mkl/include
\n-L/opt/intel/mkl/lib/intel64
\n-L/opt/intel/composer_xe_2015.2.164/compiler/lib/intel64
\n-L/usr/local/cuda-6.5/lib64
\n\nI got lots of errors like --
\n/opt/intel/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `mkl_spblas_ccoo0nhuuc__mmout_par'
\n/opt/intel/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `mkl_lapack_ps_dsymv_nb'
/opt/intel/mkl/lib/intel64/libmkl_intel_lp64.so: undefined reference to `mkl_lapack_chesvxx'
/opt/intel/mkl/lib/intel64/libmkl_intel_lp64.so: undefined reference to `mkl_lapack_ssygst'
I guess some header file is missing...
\nAppreciate your help!
", "rawMarkdown": "", "votes": null }, { "id": "63767", "postDate": "02/08/2015 00:41:08", "content": "Looks like you don't have mkl, so build it as
\nbuild.sh blas=1
\nand install blas libs possibly atlas-devel.
", "rawMarkdown": "", "votes": null }, { "id": "63768", "postDate": "02/08/2015 00:52:59", "content": "I installed intel MKL composer package and I also include MKL header and lib. Maybe something else still missing.
\nThanks
", "rawMarkdown": "", "votes": null }, { "id": "63770", "postDate": "02/08/2015 04:46:04", "content": "It was going well. I built cxxnet with a small change to the Makefile. I tested on CPU with MNIST and it worked. It took some trial and error to get the original data location correct: I put in in cxxnet/example/kaggle_bowl/bowl/data and the new images (48x48) are in cxxnet/example/kaggle_bowl/data. (the difference is the extra bowl dir).
\nThe problem is in generating the image lists:
\npython gen_img_list.py train cxxnet/example/kaggle_bowl/bowl/sampleSubmission.csv data/train train.lst
\nit immediately gives the error no such file. When I touch a file of that name there to create an empty file, it stops due to nothing in the file.
\nDid I get the data locations correct?
\nHow do you create sampleSubmission.csv? Am I putting it in the correct place?
", "rawMarkdown": "", "votes": null }, { "id": "63774", "postDate": "02/08/2015 11:45:15", "content": "Does this tutorial giving score =1.3 for everybody, It is not working for me, I had score 4.7 by this tutorial. I am doing something wrong?
", "rawMarkdown": "", "votes": null }, { "id": "63775", "postDate": "02/08/2015 13:21:55", "content": "@Bing Xu,Thank you for this great tutorial!
\nI got 1.42 on LB after a sleepless night. Totally worth it.
\nI have a GTX610 with 48 cuda core and it takes 2 hour for modeling step. Plan to upgrade to a GPU and try it again. I need to sell one kidney to buy a K80. ;)
\nStill need to figure out MKL part. :-\\
", "rawMarkdown": "", "votes": null }, { "id": "63789", "postDate": "02/09/2015 01:40:28", "content": "[quote=Charles Roberson;63770]
\nIt was going well. I built cxxnet with a small change to the Makefile. I tested on CPU with MNIST and it worked. It took some trial and error to get the original data location correct: I put in in cxxnet/example/kaggle_bowl/bowl/data and the new images (48x48) are in cxxnet/example/kaggle_bowl/data. (the difference is the extra bowl dir).
\nThe problem is in generating the image lists:
\npython gen_img_list.py train cxxnet/example/kaggle_bowl/bowl/sampleSubmission.csv data/train train.lst
\nit immediately gives the error no such file. When I touch a file of that name there to create an empty file, it stops due to nothing in the file.
\nDid I get the data locations correct?
\nHow do you create sampleSubmission.csv? Am I putting it in the correct place?
\n[/quote]
\nsampleSubmission.csv is made available by Kaggle on the data page. You point the command at wherever you store that.
", "rawMarkdown": "", "votes": null }, { "id": "63905", "postDate": "02/10/2015 09:49:59", "content": "Thanks Bing for the excellent script, I am trying to learn a bit about the CNNs that way.
\nI could get cxxnet running on cpu and without mkl and got through the example with the 48x48 resized pictures and got a score around 1.39 (after 10 hrs..). Then I tried instead of resizing with imagemagick (as in the example) to use a python script to generate 48x48 sized train files via cropping and finally stored the files using cv2.imwrite.
\nI have the same number of train images with the same 48x48 size as in the example, but generated in a different way. Now, the cxxnet file doesnt seem to learn anymore, here is a sample output what I get from cxxnet:
\n...
\nThreadImagePageIterator:image_list=./train.lst, bin=./train.bin
loading mean image from models/image_mean.bin
ThreadBufferIterator: buffer_size=2
initializing end, start working
round 0:[ 100] 467 sec elapsed[1] train-error:0.936077
round 1:[ 100] 1241 sec elapsed[2] train-error:0.936077
round 2:[ 100] 2014 sec elapsed[3] train-error:0.934785
round 3:[ 100] 2783 sec elapsed[4] train-error:0.936077
round 4:[ 100] 3566 sec elapsed[5] train-error:0.936077
round 5:[ 100] 4377 sec elapsed[6] train-error:0.934785
round 6:[ 100] 5122 sec elapsed[7] train-error:0.936077
...
Its similar to the problem description I found here:
\nhttps://github.com/antinucleon/cxxnet/issues/28
\nIs there something wrong in the way I generate the jpgs? Or should I adjust
some parameters in bowl.conf to achieve convergence? Right now I didnt touch bowl.conf,
its exactly as in the example.
@nils:
\nDid you regenerate the mean image file using the new set of training images that you created?
", "rawMarkdown": "", "votes": null }, { "id": "63978", "postDate": "02/10/2015 23:10:30", "content": "@Torgos, Thanks for the info. Of course that fixed it. I had more issues but fixed them myself and got it all working.
\n@Bing, Thanks for the work and sharing.
\nThis next suggestion can help EVERYBODY.
\n@All in CNN, The submission files are 4x to 7x bigger than the ones I used for my RF model submissions. Looking at the submission file, I see why. There are many entries with values below 0.0001 which look like 6.nnnnne-07.
\nIf all numbers less than 0.0001 (1% of 1%) were set to 0, that would significantly cut the data size: likely by 5x or better. This will translate into much smaller zip files as well. My current CNN submission zip files are 70 MB while my RF submission zip files are around 10MB to 15MB.
\nI'd do this myself, but I don't know Python and it looks like line 29 in the make_submission.py file needs changing.
\nI think doing this will help everybody and kaggle.com due to a reduction in bandwidth. Things are just going to heat up more in the next 34 days! Also, there are some other contests ending near the same day as this one.
", "rawMarkdown": "", "votes": null }, { "id": "64133", "postDate": "02/12/2015 18:15:57", "content": "Hi all,
\nI'm new to this forum (and not a data science expert either). First of all, I want to thank Bing Xu & Tianqi Chen for this nice cxxnet thing. It took me several unsuccessful days to compile opencv on my old 2009 Macbook (with Mavericks) until I realized I could use homebrew; the cxxnet path has been (quite) smooth since then; I am now where many of you were 40-50 days ago: score of 1.39, thinking on input augmentation...
\nMy problem is that my GPU has not enough memory (just 256MB) to run cxxnet with the given bowl.conf, and it took 10 hours to run 45 rounds in CPU mode.
\nIs there something I can try to run cxxnet in my GPU? I have reduced the resize parameters from 48*48 to 30*30, but still no luck. (BTW, I know GPU+CUDA works properly because the MNIST tutorial runs on it)
\nThanks!
", "rawMarkdown": "", "votes": null }, { "id": "64134", "postDate": "02/12/2015 18:21:48", "content": "Try reducing the batch size. Either way I recommend getting ec2 g2large instance. Spot requests are $0.10 per hour.
", "rawMarkdown": "", "votes": null }, { "id": "64135", "postDate": "02/12/2015 18:31:11", "content": "Thanks for your fast answer. I took a look to aws and google cloud but I did not understand how to proceed for this specific need: do you upload the code as-is to a virtual machine? or do you have to adapt the code to a virtual array of GPUs?
", "rawMarkdown": "", "votes": null }, { "id": "64136", "postDate": "02/12/2015 18:37:45", "content": "You gain access to your own instance. You can log and do everything with it through ssh.
\nWhat I did was
\n1. Create instance (ubuntu)
\n2. Attach ebs storage to it for persistant data
\n3. install everything : libraries, cuda. Save the image (ami) of the instance (it can be terminated any time so you need to make sure you can start it up again fast).
", "rawMarkdown": "", "votes": null }, { "id": "64156", "postDate": "02/13/2015 05:39:22", "content": "You can follow this blog or use the AMI they mentioned
\nhttp://tleyden.github.io/blog/2014/10/25/cuda-6-dot-5-on-aws-gpu-instance-running-ubuntu-14-dot-04/
\nUse spot instances. GPU spot instance cost is around 0.064
", "rawMarkdown": "", "votes": null }, { "id": "64195", "postDate": "02/13/2015 21:59:15", "content": "[quote=zeros_benchmark;63973]
\n@nils:
\nDid you regenerate the mean image file using the new set of training images that you created?
\n[/quote]
\nI deleted all the files in the models folder so I assume the mean image was newly generated. I now also got cxxnet working on my image set-I simply did a clean compilation and generated all the cropped images again. I still dont know what went wrong before but it works now, thanks!
\nAnother thing, is it possible to view the filter banks within cxxnet? Is there some simple way to extract and visualize them from the models files?
", "rawMarkdown": "", "votes": null }, { "id": "65089", "postDate": "02/28/2015 01:58:09", "content": "When I try to use the code in cv, the prediction file sometimes lost some samples.
\nSay the test.lst have 10200 samples, which is correct, but test.txt only contains 10000 samples. How to deal with it?
\nSorry I didn't go through every page of this post. maybe my question is answered somewhere.
", "rawMarkdown": "", "votes": null }, { "id": "65090", "postDate": "02/28/2015 02:05:15", "content": "Change the batch size; cxxnet only predicts multiples of it.
\n\nhttps://github.com/antinucleon/cxxnet/issues/17
", "rawMarkdown": "", "votes": null }, { "id": "65091", "postDate": "02/28/2015 02:11:30", "content": "[quote=Torgos;65090]
\nChange the batch size; cxxnet only predicts multiples of it.
\nhttps://github.com/antinucleon/cxxnet/issues/17
\n[/quote]
\nI see. Thank you very much!
", "rawMarkdown": "", "votes": null }, { "id": "65092", "postDate": "02/28/2015 02:16:13", "content": "We will release v0.2 of CXXNET. It will fully fix this problem. And new version will support real time augmentation, multi-GPU training and distributed training. Stay tuned!
\n\n[quote=rcarson;65089]
\nWhen I try to use the code in cv, the prediction file sometimes lost some samples.
\nSay the test.lst have 10200 samples, which is correct, but test.txt only contains 10000 samples. How to deal with it?
\nSorry I didn't go through every page of this post. maybe my question is answered somewhere.
\n[/quote]
", "rawMarkdown": "", "votes": null }, { "id": "65098", "postDate": "02/28/2015 03:00:19", "content": "[quote=Bing Xu;65092]
\nWe will release v0.2 of CXXNET. It will fully fix this problem. And new version will support real time augmentation, multi-GPU training and distributed training. Stay tuned!
\n[/quote]
\n\nThat's awesome! I'm on the edge of my seat :)
\nWhat sort of real time augmentation are you going to include?
", "rawMarkdown": "", "votes": null }, { "id": "65112", "postDate": "02/28/2015 06:17:12", "content": "[quote=Bing Xu;65092]
\nWe will release v0.2 of CXXNET. It will fully fix this problem. And new version will support real time augmentation, multi-GPU training and distributed training. Stay tuned!
\n[quote=rcarson;65089]
\nWhen I try to use the code in cv, the prediction file sometimes lost some samples.
\nSay the test.lst have 10200 samples, which is correct, but test.txt only contains 10000 samples. How to deal with it?
\nSorry I didn't go through every page of this post. maybe my question is answered somewhere.
\n[/quote]
\n[/quote]
\nThat's terrific Bing. You got all the missing pieces covered with those updates.
", "rawMarkdown": "", "votes": null }, { "id": "65211", "postDate": "03/02/2015 02:55:28", "content": "", "rawMarkdown": "", "votes": null }, { "id": "65212", "postDate": "03/02/2015 02:56:37", "content": "[quote=Robert Griesmeyer;62324]
\nDid anyone get it running on the Macbook on OSX?
\n[/quote]
\nYes, use Blas and the following attached files for generating cxxnet and your images. Otherwise, you will get a mismatch in the shape of tensors. Total running time for training on a MacAir (CPU only) was ~5 hours with prediction time of about 15 minutes.
\nRegards,
\nMark
", "rawMarkdown": "", "votes": null }, { "id": "65401", "postDate": "03/04/2015 11:47:27", "content": "get the following error when running ./build.sh.
cxxnet/io/cxxnet_data.cpp:6:28: error: mshadow/tensor.h: No such file or directory
In file included from cxxnet/io/cxxnet_data.cpp:18:
cxxnet/io/cxxnet_iter_cifar-inl.hpp:5:38: error: mshadow/tensor_container.h: No such file or directory
In file included from cxxnet/io/cxxnet_data.cpp:21:
cxxnet/io/cxxnet_iter_img-inl.hpp:14:30: error: opencv2/opencv.hpp: No such file or directory
This means that the header files are not being included correctly. I have modified the Makefile with -I option to include all the headers. Does anybody have any idea why this is happening?
\nI have already cloned mshadow and can see the header files in the directory but they are not being included for some reason.
\nPlease find attached my Makefile and error log.Thanks!
", "rawMarkdown": "", "votes": null }, { "id": "65402", "postDate": "03/04/2015 11:50:43", "content": "^ Oops, The last Makefile(1.57 KB) is the correct one. Ignore the first one.
", "rawMarkdown": "", "votes": null }, { "id": "65559", "postDate": "03/06/2015 02:17:26", "content": "Hi Bing, thanx a lot for sharing! In the sample config file (bowl.conf), some conv layers don't have a stride value. So I'm wondering what the default stride is.
\nPlease advise. Many thanx in advance!
\n\nJJ
", "rawMarkdown": "", "votes": null }, { "id": "65627", "postDate": "03/06/2015 19:53:08", "content": "[quote=Bing Xu;65092]
\nWe will release v0.2 of CXXNET. It will fully fix this problem. And new version will support real time augmentation, multi-GPU training and distributed training. Stay tuned!
\n[/quote]
\nLooking forward to it for the diabetes competition; I don't suppose you're including a quick option in the config file to split (and report on) a % of data off for validation?
", "rawMarkdown": "", "votes": null }, { "id": "65631", "postDate": "03/06/2015 20:01:30", "content": "There is many ways to do it and we don't think it is an important feature to hard code.
\nSolution in V2:
\n1. sed -n '1, 20000p' train.lst > tr.lst
\nsed -n '20000, 30337p' train.lst > va.lst
\nThen generate bin according to it.
\n2. Use script in tools to split list file in to parts, like 6 separate bin files. Then set in configuration file which bins you want to use to train and which bins you want to treat as eval
\n\n3. Use python interface, do slice by numpy
\n[quote=Torgos;65627]
\n[quote=Bing Xu;65092]
\nWe will release v0.2 of CXXNET. It will fully fix this problem. And new version will support real time augmentation, multi-GPU training and distributed training. Stay tuned!
\n[/quote]
\nLooking forward to it for the diabetes competition; I don't suppose you're including a quick option in the config file to split (and report on) a % of data off for validation?
\n[/quote]
", "rawMarkdown": "", "votes": null }, { "id": "65677", "postDate": "03/07/2015 11:54:39", "content": "I was able to get all the header files included but running ./build.sh is throwing errors like:
\ncxxnet/io/cxxnet_data.cpp:12: error: ‘real_t’ in namespace ‘mshadow’ does not name a type
\ncxxnet/io/cxxnet_iter_cifar-inl.hpp:14: error: ‘struct mshadow::Tensor<mshadow::cpu, 4, float>’ has no member named ‘dptr’
\ncxxnet/io/cxxnet_iter_cifar-inl.hpp:42: error: ‘struct mshadow::Tensor<mshadow::cpu, 4, float>’ has no member named ‘shape’
\nPlease find my Makefile attached. I have added all of the header file directories for OpenCV with the -I flag to CFLAGS because I had installed OpenCV in a custom location (no root access :/ ).
\n
The full error log is present in whatever.log
@Bing, can you take a quick look at this? Btw, thanks for writing CXXNET :)
", "rawMarkdown": "", "votes": null }, { "id": "65684", "postDate": "03/07/2015 15:45:38", "content": "[quote=Saurav Gupta;65677]
\nI was able to get all the header files included but running ./build.sh is throwing errors like:
\ncxxnet/io/cxxnet_data.cpp:12: error: ‘real_t’ in namespace ‘mshadow’ does not name a type
\ncxxnet/io/cxxnet_iter_cifar-inl.hpp:14: error: ‘struct mshadow::Tensor<mshadow::cpu, 4, float>’ has no member named ‘dptr’
\ncxxnet/io/cxxnet_iter_cifar-inl.hpp:42: error: ‘struct mshadow::Tensor<mshadow::cpu, 4, float>’ has no member named ‘shape’
\nPlease find my Makefile attached. I have added all of the header file directories for OpenCV with the -I flag to CFLAGS because I had installed OpenCV in a custom location (no root access :/ ).
\n
The full error log is present in whatever.log
@Bing, can you take a quick look at this? Btw, thanks for writing CXXNET :)
\n[/quote]
\nYou may be using the newer version of the mshadow code (2.x). You have to point to the older version (1.x).
", "rawMarkdown": "", "votes": null }, { "id": "65771", "postDate": "03/09/2015 11:39:08", "content": "deleted
", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 60429, "author_name": "blackeagle", "author_url": "", "post_date": "12/17/2014 05:03:41", "content": "Can I use CXXNET on a windows machine?
", "votes": null, "replies": [] }, { "id": 60431, "author_name": "binghsu", "author_url": "", "post_date": "12/17/2014 05:06:18", "content": "I don't have Windows machine with GPU so I am not very sure.. I guess you can but you need to configure by yourself.
\n[quote=kartal;60429]
\nCan I use CXXNET on a windows machine?
\n[/quote]
", "votes": null, "replies": [] }, { "id": 60439, "author_name": "issac4sacrifice", "author_url": "", "post_date": "12/17/2014 07:20:29", "content": "Thank you for your share.
\nBut I want to know, if I can run CXXNET with CPU?
\nI just have a macbook with intel GPU...
", "votes": null, "replies": [] }, { "id": 60440, "author_name": "binghsu", "author_url": "", "post_date": "12/17/2014 07:31:27", "content": "You can, you need blas or MKL installed. Then set dev=cpu in config file
\n\n[quote=Issac;60439]
\nThank you for your share.
\nBut I want to know, if I can run CXXNET with CPU?
\nI just have a macbook with intel GPU...
\n[/quote]
", "votes": null, "replies": [] }, { "id": 60441, "author_name": "phunter", "author_url": "", "post_date": "12/17/2014 07:46:28", "content": "Very nice, star-ed it on github!
", "votes": null, "replies": [] }, { "id": 60464, "author_name": "jiweiliu", "author_url": "", "post_date": "12/17/2014 11:13:08", "content": "Thank you very much! How much main memory and GPU memory does it require?
", "votes": null, "replies": [] }, { "id": 60483, "author_name": "", "author_url": "", "post_date": "12/17/2014 14:12:56", "content": "What's the difference between cxxnet and Caffe?
", "votes": null, "replies": [] }, { "id": 60499, "author_name": "adjgiulio", "author_url": "", "post_date": "12/17/2014 16:30:48", "content": "[quote=Bing Xu;60426]
\nHi all,
\nI just made a guide of how to use conv net to achieve NLL of 1.382285. On a modern GPU, it only takes 5 minute to get this result.
\nPlease check https://github.com/antinucleon/cxxnet/tree/master/example/kaggle_bowl , and don't forget to star CXXNET :)
\nPS. I made it in a hurry, so if there is any problem please let me know!
\n[/quote]
\nBing, thanks! This is great work, as usual :-)
\nI hate to tell you this, but in the beginning I'm afraid you'll end up getting more questions about how to install this (and errors while installing) than how to use it. I think adding more details in the wiki about the install, dependencies, where to get the dependencies, how to make sure those work, how to troubleshoot errors,.. might save you time in the long run.
\nBut again, awesome work! Thanks a lot!
", "votes": null, "replies": [] }, { "id": 60503, "author_name": "abhishek", "author_url": "", "post_date": "12/17/2014 16:43:42", "content": "Ahh... I wish I had GPU :-/
", "votes": null, "replies": [] }, { "id": 61418, "author_name": "johnnystrings", "author_url": "", "post_date": "12/17/2014 20:02:17", "content": "Thanks Bing - this is as good and straight-forward of an intro to deep learning using a GPU as anyone could hope for. I understand you had a GTX 780M or something like that for processing. May I also ask how much RAM your system had, and how much was in use while processing, if you happen to know? Thanks!
", "votes": null, "replies": [] }, { "id": 61419, "author_name": "binghsu", "author_url": "", "post_date": "12/17/2014 20:09:22", "content": "300MB GPU mem, 1GB CPU mem is enough. I remember
\n[quote=Johnny Strings;61418]
\nThanks Bing - this is as good and straight-forward of an intro to deep learning using a GPU as anyone could hope for. I understand you had a GTX 780M or something like that for processing. May I also ask how much RAM your system had, and how much was in use while processing, if you happen to know? Thanks!
\n[/quote]
", "votes": null, "replies": [] }, { "id": 61420, "author_name": "binghsu", "author_url": "", "post_date": "12/17/2014 20:09:52", "content": "You can use CPU. MKL will give you reasonable performance
\n[quote=Abhishek;60503]
\nAhh... I wish I had GPU :-/
\n[/quote]
", "votes": null, "replies": [] }, { "id": 61421, "author_name": "binghsu", "author_url": "", "post_date": "12/17/2014 20:10:50", "content": "Thanks for this advice, I will put link of wiki to README
\n[quote=Giulio;60499]
\n[quote=Bing Xu;60426]
\nHi all,
\nI just made a guide of how to use conv net to achieve NLL of 1.382285. On a modern GPU, it only takes 5 minute to get this result.
\nPlease check https://github.com/antinucleon/cxxnet/tree/master/example/kaggle_bowl , and don't forget to star CXXNET :)
\nPS. I made it in a hurry, so if there is any problem please let me know!
\n[/quote]
\nBing, thanks! This is great work, as usual :-)
\nI hate to tell you this, but in the beginning I'm afraid you'll end up getting more questions about how to install this (and errors while installing) than how to use it. I think adding more details in the wiki about the install, dependencies, where to get the dependencies, how to make sure those work, how to troubleshoot errors,.. might save you time in the long run.
\nBut again, awesome work! Thanks a lot!
\n[/quote]
", "votes": null, "replies": [] }, { "id": 61423, "author_name": "dylanjf", "author_url": "", "post_date": "12/17/2014 20:56:13", "content": "thanks for sharing. i'm attempting to get this setup with caffe, so this will be a good guide. i'll share my version too if i get it working :)
", "votes": null, "replies": [] }, { "id": 61426, "author_name": "tqchen", "author_url": "", "post_date": "12/17/2014 21:21:27", "content": "I am one of the author. In short, they are similar. CXXNET is much shorter, and concise in terms of lines of code.
\n\n[quote=ziyuang;60483]
\nWhat's the difference between cxxnet and Caffe?
\n[/quote]
", "votes": null, "replies": [] }, { "id": 61427, "author_name": "mikeskim", "author_url": "", "post_date": "12/17/2014 21:22:31", "content": "[quote=Abhishek;60503]
\nAhh... I wish I had GPU :-/
\n[/quote]
\nHave you considered using AWS GPU instances? http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using_cluster_computing.html
If you're okay with spot pricing and sudden cutoffs, I'm sure you can get a good rate.
[quote=Tianqi Chen;61426]
\nI am one of the author. In short, they are similar. CXXNET is much shorter, and concise in terms of lines of code.
\n[quote=ziyuang;60483]
\nWhat's the difference between cxxnet and Caffe?
\n[/quote]
\n[/quote]
\nHow about from the user's angle? Does it mean CXXNET will be faster, or does it mean it will be easier to learn because of the conciseness?
", "votes": null, "replies": [] }, { "id": 61432, "author_name": "clustifier", "author_url": "", "post_date": "12/17/2014 22:09:44", "content": "Can CXXNET be run on Macbook pro with GPU?
\nI'm getting this:
\nIn file included from ./mshadow/mshadow/tensor.h:11:
./mshadow/mshadow/tensor_base.h:76:12: fatal error: 'mkl.h' file not found
#include <mkl.h>
^
1 error generated.
make: *** [cxxnet_data.o] Error 1
In default it is using MKL. I don't have Macbook so I am not sure. Ideally it is able to run on all NVIDIA GPU.
\nPlease refer https://github.com/antinucleon/cxxnet#build-guide to build it.
\n[quote=clustifier;61432]
\nCan CXXNET be run on Macbook pro with GPU?
\nI'm getting this:
\nIn file included from ./mshadow/mshadow/tensor.h:11:
./mshadow/mshadow/tensor_base.h:76:12: fatal error: 'mkl.h' file not found
#include <mkl.h>
^
1 error generated.
make: *** [cxxnet_data.o] Error 1
[/quote]
", "votes": null, "replies": [] }, { "id": 61436, "author_name": "jason50", "author_url": "", "post_date": "12/17/2014 23:36:42", "content": "Thanks for sharing the code. I have a problems when installing cxxnet on Ubuntu 14.04
\n$ build.sh blas=1
\ng++ -Wall -g -O3 -msse3 -Wno-unknown-pragmas -funroll-loops -I./mshadow/ -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_CBLAS=1 -o bin/cxxnet cxxnet/cxxnet_main.cpp cxxnet_data.o cxxnet_nnet_cpu.o cxxnet_nnet_gpu.o -lm -lcudart -lcublas -lcurand -lz `pkg-config --libs opencv` -lblas
/usr/bin/ld: cxxnet_data.o: undefined reference to symbol 'sem_post@@GLIBC_2.2.5'
//lib/x86_64-linux-gnu/libpthread.so.0: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
make: *** [bin/cxxnet] Error 1
How could I fix it?
\n\nMeanwhile, do you mean add 'dev=cpu' to bowl.conf ? I didn't find 'dev=cpu' or 'dev=gpu' in any fil
\ne. I guess that you are using GPU and it is the default option.
\n\nMany Thanks
\n\n[
\nquote=Bing Xu;60440]
\nYou can, you need blas or MKL installed. Then set dev=cpu in config file
\n[quote=Issac;60439]
\nThank you for your share.
\nBut I want to know, if I can run CXXNET with CPU?
\nI just have a macbook with intel GPU...
\n[/quote]
\n[/quote]
", "votes": null, "replies": [] }, { "id": 61437, "author_name": "binghsu", "author_url": "", "post_date": "12/17/2014 23:40:12", "content": "I suggest you post in issue on Github about building problem. It is about your pthread problem on your machine. I guess may be upgrade your gcc will help. But I am not sure because I never use Mac.
\n\n\nYou can add ```dev = cpu``` at global setting part, I mean no indent
\n\n[quote=Jason;61436]
\nThanks for sharing the code. I have a problems when installing cxxnet on Ubuntu 14.04
\n$ build.sh blas=1
\ng++ -Wall -g -O3 -msse3 -Wno-unknown-pragmas -funroll-loops -I./mshadow/ -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_CBLAS=1 -o bin/cxxnet cxxnet/cxxnet_main.cpp cxxnet_data.o cxxnet_nnet_cpu.o cxxnet_nnet_gpu.o -lm -lcudart -lcublas -lcurand -lz `pkg-config --libs opencv` -lblas
/usr/bin/ld: cxxnet_data.o: undefined reference to symbol 'sem_post@@GLIBC_2.2.5'
//lib/x86_64-linux-gnu/libpthread.so.0: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
make: *** [bin/cxxnet] Error 1
How could I fix it?
\nMeanwhile, do you mean add 'dev=cpu' to bowl.conf ? I didn't find 'dev=cpu' or 'dev=gpu' in any fil
\ne. I guess that you are using GPU and it is the default option.
\nMany Thanks
\n[
\nquote=Bing Xu;60440]
\nYou can, you need blas or MKL installed. Then set dev=cpu in config file
\n[quote=Issac;60439]
\nThank you for your share.
\nBut I want to know, if I can run CXXNET with CPU?
\nI just have a macbook with intel GPU...
\n[/quote]
\n[/quote]
\n[/quote]
", "votes": null, "replies": [] }, { "id": 61438, "author_name": "jason50", "author_url": "", "post_date": "12/18/2014 00:25:02", "content": "Many Thanks Bing,
\nI give an updated details of my building environment at GitHub issue 14
\nCheers.
\n\n[quote=Bing Xu;61437]
\nI suggest you post in issue on Github about building problem. It is about your pthread problem on your machine. I guess may be upgrade your gcc will help. But I am not sure because I never use Mac.
\nYou can add ```dev = cpu``` at global setting part, I mean no indent
\n[quote=Jason;61436]
\nThanks for sharing the code. I have a problems when installing cxxnet on Ubuntu 14.04
\n$ build.sh blas=1
\ng++ -Wall -g -O3 -msse3 -Wno-unknown-pragmas -funroll-loops -I./mshadow/ -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_CBLAS=1 -o bin/cxxnet cxxnet/cxxnet_main.cpp cxxnet_data.o cxxnet_nnet_cpu.o cxxnet_nnet_gpu.o -lm -lcudart -lcublas -lcurand -lz `pkg-config --libs opencv` -lblas
/usr/bin/ld: cxxnet_data.o: undefined reference to symbol 'sem_post@@GLIBC_2.2.5'
//lib/x86_64-linux-gnu/libpthread.so.0: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
make: *** [bin/cxxnet] Error 1
How could I fix it?
\nMeanwhile, do you mean add 'dev=cpu' to bowl.conf ? I didn't find 'dev=cpu' or 'dev=gpu' in any fil
\ne. I guess that you are using GPU and it is the default option.
\nMany Thanks
\n[
\nquote=Bing Xu;60440]
\nYou can, you need blas or MKL installed. Then set dev=cpu in config file
\n[quote=Issac;60439]
\nThank you for your share.
\nBut I want to know, if I can run CXXNET with CPU?
\nI just have a macbook with intel GPU...
\n[/quote]
\n[/quote]
\n[/quote]
\n[/quote]
", "votes": null, "replies": [] }, { "id": 61442, "author_name": "blackeagle", "author_url": "", "post_date": "12/18/2014 03:23:38", "content": "Hi Bing,
\nWhen I run the gen_train.py function I am getting these warnings "convert is not recognized as an internal or external command". Are you using a special python package to resize images?
", "votes": null, "replies": [] }, { "id": 61443, "author_name": "binghsu", "author_url": "", "post_date": "12/18/2014 03:25:08", "content": "You need install http://www.imagemagick.org/ to use convert
\n[quote=kartal;61442]
\nHi Bing,
\nWhen I run the gen_train.py function I am getting these warnings "convert is not recognized as an internal or external command". Are you using a special python package to resize images?
\n[/quote]
", "votes": null, "replies": [] }, { "id": 61446, "author_name": "blackeagle", "author_url": "", "post_date": "12/18/2014 04:39:41", "content": "OK thanks I did get gen_train.py to work but when I look at the converted images the original sizes are preserved. They are not 48x48 as suggested.
", "votes": null, "replies": [] }, { "id": 61447, "author_name": "blackeagle", "author_url": "", "post_date": "12/18/2014 05:07:59", "content": "OK I figured out the problem. Maybe this is due to I am using MinGW in Windows but the correct form of the python code should be as below in order for the resizing to work.
\n\nimport os
import sys
import subprocess
if len(sys.argv) < 3:
print "Usage: python gen_train.py input_folder output_folder"
exit(1)
fi = sys.argv[1]
fo = sys.argv[2]
cmd = "convert "
classes = os.listdir(fi)
os.chdir(fo)
for cls in classes:
try:
os.mkdir(cls)
except:
pass
imgs = os.listdir(fi + cls)
for img in imgs:
md = ""
md += cmd
md += fi + cls + "/" + img
md += " -resize 48x48!"
md += " " + fo + cls + "/" + img
os.system(md)
Thanks Bing, I just got it working and submitted results.
\nFor others - I'm using Ubuntu 14.04, and had to muck around with the CXXNet Makefiles a bit, adding -I and -L references to the CUDA library locations.
", "votes": null, "replies": [] }, { "id": 61497, "author_name": "jfkingiii", "author_url": "", "post_date": "12/18/2014 17:12:16", "content": "[quote=Bing Xu;60426]
\nHi all,
\nI just made a guide of how to use conv net to achieve NLL of 1.382285.
\n[/quote]
\n\nBefore I try to install it, can it be expected to work on an NVIDIA GRID K520? This is an older model with compute capability 3.0.
", "votes": null, "replies": [] }, { "id": 61503, "author_name": "terrychenism", "author_url": "", "post_date": "12/18/2014 20:45:31", "content": "Hi, Bing,
\nI follow your net config file and create a caffe proto file as attached, but it cannot get your result. Is there any difference between your cxxnet and caffe?
", "votes": null, "replies": [] }, { "id": 61522, "author_name": "clustifier", "author_url": "", "post_date": "12/19/2014 06:10:19", "content": "Hi Bing,
\nAfter compiling successfully for cpu and adding the dev=cpu I'm getting this:
\n$ ../../bin/cxxnet bowl.conf
CXXNetTrainer, devCPU=1
ConvolutionLayer: nstep=256
ConvolutionLayer: nstep=256
ConvolutionLayer: nstep=128
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
node[0].shape: 256,3,48,48
node[1].shape: 256,96,12,12
node[2].shape: 256,96,12,12
node[3].shape: 256,96,6,6
node[4].shape: 256,128,8,8
node[5].shape: 256,128,8,8
node[6].shape: 256,128,8,8
node[7].shape: 256,128,8,8
node[8].shape: 256,128,4,4
node[9].shape: 1,1,256,2048
node[10].shape: 1,1,256,512
node[11].shape: 1,1,256,512
node[12].shape: 1,1,256,512
node[13].shape: 1,1,256,512
node[14].shape: 1,1,256,121
ThreadImagePageIterator:image_list=./train.lst, bin=./train.bin
cannot find models/image_mean.bin: create mean image, this will take some time...
[ 9000] images processed, 1 sec elapsedError:decoding fail
Can you please see what I might do wrong?
\nThanks,
\nC
", "votes": null, "replies": [] }, { "id": 61523, "author_name": "binghsu", "author_url": "", "post_date": "12/19/2014 06:12:01", "content": "It means opencv can't decode image in the binary file. I suggest you check your opencv and process in generating binary file.
\n[quote=clustifier;61522]
\nHi Bing,
\nAfter compiling successfully for cpu and adding the dev=cpu I'm getting this:
\n$ ../../bin/cxxnet bowl.conf
CXXNetTrainer, devCPU=1
ConvolutionLayer: nstep=256
ConvolutionLayer: nstep=256
ConvolutionLayer: nstep=128
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
node[0].shape: 256,3,48,48
node[1].shape: 256,96,12,12
node[2].shape: 256,96,12,12
node[3].shape: 256,96,6,6
node[4].shape: 256,128,8,8
node[5].shape: 256,128,8,8
node[6].shape: 256,128,8,8
node[7].shape: 256,128,8,8
node[8].shape: 256,128,4,4
node[9].shape: 1,1,256,2048
node[10].shape: 1,1,256,512
node[11].shape: 1,1,256,512
node[12].shape: 1,1,256,512
node[13].shape: 1,1,256,512
node[14].shape: 1,1,256,121
ThreadImagePageIterator:image_list=./train.lst, bin=./train.bin
cannot find models/image_mean.bin: create mean image, this will take some time...
[ 9000] images processed, 1 sec elapsedError:decoding fail
Can you please see what I might do wrong?
\nThanks,
\nC
\n[/quote]
", "votes": null, "replies": [] }, { "id": 61528, "author_name": "brundage", "author_url": "", "post_date": "12/19/2014 08:39:46", "content": "It seems you need either Intel MKL or NVIDIA CUDA installed to use this?
\nbuild.sh blas=1 =>
\n./mshadow/mshadow/tensor_base.h:83:12: fatal error: 'cublas.h' file not found
\nMSHADOW_USE_CUDA=0 ./build.sh blas=1 =>
\n./mshadow/mshadow/tensor_base.h:282:13: error: use of undeclared identifier 'exit'
", "votes": null, "replies": [] }, { "id": 61575, "author_name": "gregoirel", "author_url": "", "post_date": "12/20/2014 14:48:42", "content": "[quote=Michael Bewley;61471]
\nThanks Bing, I just got it working and submitted results.
\nFor others - I'm using Ubuntu 14.04, and had to muck around with the CXXNet Makefiles a bit, adding -I and -L references to the CUDA library locations.
\n[/quote]
\n\nMichael, I followed the same path you took but unfortunately, when I run cxxnet bowl.conf, I get a segmentation fault.
\nWould you care to confirm the md5sum for the 2 bin files? I get :
\na96b2dc558eb9662b417d9b23ca0fdd4 train.bin
\n891b7e1072d146f850d43cad8b0dd808 test.bin
\nThanks!
", "votes": null, "replies": [] }, { "id": 61581, "author_name": "emolson", "author_url": "", "post_date": "12/20/2014 16:07:01", "content": "[quote=gregl;61575]
\n[quote=Michael Bewley;61471]
\nThanks Bing, I just got it working and submitted results.
\nFor others - I'm using Ubuntu 14.04, and had to muck around with the CXXNet Makefiles a bit, adding -I and -L references to the CUDA library locations.
\n[/quote]
\nMichael, I followed the same path you took but unfortunately, when I run cxxnet bowl.conf, I get a segmentation fault.
\nWould you care to confirm the md5sum for the 2 bin files? I get :
\na96b2dc558eb9662b417d9b23ca0fdd4 train.bin
\n891b7e1072d146f850d43cad8b0dd808 test.bin
\nThanks!
\n[/quote]
\nFor me:
\n50cd388bc640b63e2144b551fb6260fd test.bin
9f847cc39b7fee93e842a668cd4e72b4 train.bin
for me:
\nff0bee47c19b6c7ab79b29d3c37872b2
79343a11c7f80cd3b8e94712ae42cdc8
I realize the checksum of the bin file is probably irrelevant because machine dependent...
\nAnyway I fixed the issue and was able to get the 1.38 score. It seems my problem was to have built opencv before installing CUDA.
", "votes": null, "replies": [] }, { "id": 61599, "author_name": "clustifier", "author_url": "", "post_date": "12/20/2014 19:56:25", "content": "[quote=gregl;61598]
\nI realize the checksum of the bin file is probably irrelevant because machine dependent...
\n[/quote]
\nI don't think it is true.
\nWhat is your md5 checksums now?
", "votes": null, "replies": [] }, { "id": 61600, "author_name": "gregoirel", "author_url": "", "post_date": "12/20/2014 20:28:29", "content": "[quote=clustifier;61599]
\n[quote=gregl;61598]
\nI realize the checksum of the bin file is probably irrelevant because machine dependent...
\n[/quote]
\nI don't think it is true.
\nWhat is your md5 checksums now?
\n[/quote]
\n\nSame as before. im2bin was working all along for me, only cxxnet was not.
", "votes": null, "replies": [] }, { "id": 61601, "author_name": "clustifier", "author_url": "", "post_date": "12/20/2014 21:18:16", "content": "thanks.
\nbut why our md5's are different?
", "votes": null, "replies": [] }, { "id": 61632, "author_name": "gregoirel", "author_url": "", "post_date": "12/21/2014 19:34:01", "content": "[quote=clustifier;61601]
\nthanks.
\nbut why our md5's are different?
\n[/quote]
\n\nI built im2bin with blas, not the default mlk.
\nAlso, upstream, the md5sum of train.lst is:
\n399ab27e818cabcb5bd2158c15451a80 train.lst
\nwith the first 5 lines:
\n3406 10 data/train/chaetognath_non_sagitta/23677.jpg
22212 90 data/train/radiolarian_chain/157296.jpg
19772 83 data/train/protist_fuzzy_olive/147898.jpg
23435 98 data/train/siphonophore_calycophoran_rocketship_young/36262.jpg
18710 72 data/train/hydromedusae_solmaris/47760.jpg
My train.lst is different. I think the issue is that os.listdir does not guarantee any ordering of the results.
\n[quote=gregl;61632]
\n[quote=clustifier;61601]
\nthanks.
\nbut why our md5's are different?
\n[/quote]
\nI built im2bin with blas, not the default mlk.
\nAlso, upstream, the md5sum of train.lst is:
\n399ab27e818cabcb5bd2158c15451a80 train.lst
\nwith the first 5 lines:
\n3406 10 data/train/chaetognath_non_sagitta/23677.jpg
22212 90 data/train/radiolarian_chain/157296.jpg
19772 83 data/train/protist_fuzzy_olive/147898.jpg
23435 98 data/train/siphonophore_calycophoran_rocketship_young/36262.jpg
18710 72 data/train/hydromedusae_solmaris/47760.jpg
[/quote]
", "votes": null, "replies": [] }, { "id": 61726, "author_name": "nissanpow", "author_url": "", "post_date": "12/23/2014 01:56:21", "content": "Include stdlib.h in tensor_base.h to fix the error about 'exit'.
\n\n\n[quote=Synthient;61528]
\nIt seems you need either Intel MKL or NVIDIA CUDA installed to use this?
\nbuild.sh blas=1 =>
\n./mshadow/mshadow/tensor_base.h:83:12: fatal error: 'cublas.h' file not found
\nMSHADOW_USE_CUDA=0 ./build.sh blas=1 =>
\n./mshadow/mshadow/tensor_base.h:282:13: error: use of undeclared identifier 'exit'
\n[/quote]
", "votes": null, "replies": [] }, { "id": 61732, "author_name": "nissanpow", "author_url": "", "post_date": "12/23/2014 05:19:24", "content": "Hi, this is quite interesting. CXXNET was way easier to setup compared to caffe. I have some quick questions about CXXNET though:
\n- is it possible to resume training from a saved model file?
\n- is it possible to extract features from a particular layer?
\n- is there some documentation about what the various parameters in the config file mean? eg. wmat:wd
\n- how is the performance of cxxnet vs caffe?
\n\n\n[quote=Bing Xu;60426]
\nHi all,
\nI just made a guide of how to use conv net to achieve NLL of 1.382285. On a modern GPU, it only takes 5 minute to get this result.
\nPlease check https://github.com/antinucleon/cxxnet/tree/master/example/kaggle_bowl , and don't forget to star CXXNET :)
\nCXXNET is yet another neural network toolkit. It is able to run on NVIDIA GPU or CPU. If you want to run on CPU, you only need OpenCV & MKL(or OpenBlas/ATLAS) installed. No other dependence is required.
\nPS. I made it in a hurry, so if there is any problem please let me know!
\n[/quote]
", "votes": null, "replies": [] }, { "id": 61733, "author_name": "binghsu", "author_url": "", "post_date": "12/23/2014 05:53:49", "content": "Please refer to the wiki on Github https://github.com/antinucleon/cxxnet/wiki
\n[quote=Nissan Pow;61732]
\nHi, this is quite interesting. CXXNET was way easier to setup compared to caffe. I have some quick questions about CXXNET though:
\n- is it possible to resume training from a saved model file?
\n- is it possible to extract features from a particular layer?
\n- is there some documentation about what the various parameters in the config file mean? eg. wmat:wd
\n- how is the performance of cxxnet vs caffe?
\n[quote=Bing Xu;60426]
\nHi all,
\nI just made a guide of how to use conv net to achieve NLL of 1.382285. On a modern GPU, it only takes 5 minute to get this result.
\nPlease check https://github.com/antinucleon/cxxnet/tree/master/example/kaggle_bowl , and don't forget to star CXXNET :)
\nCXXNET is yet another neural network toolkit. It is able to run on NVIDIA GPU or CPU. If you want to run on CPU, you only need OpenCV & MKL(or OpenBlas/ATLAS) installed. No other dependence is required.
\nPS. I made it in a hurry, so if there is any problem please let me know!
\n[/quote]
\n[/quote]
", "votes": null, "replies": [] }, { "id": 61783, "author_name": "clustifier", "author_url": "", "post_date": "12/23/2014 22:04:51", "content": "Has someone used CXXNET on mac osx successfully?
\n(with or without cuda)
", "votes": null, "replies": [] }, { "id": 61786, "author_name": "mconway", "author_url": "", "post_date": "12/23/2014 22:18:49", "content": "[quote=clustifier;61783]
\nHas someone used CXXNET on mac osx successfully?
\n(with or without cuda)
\n[/quote]
\nNope, trying though... can't seem to get by the includes. The file cblas.h is actually part of the Accelerate framework on the Mac, but now we are trying to find cublas.h. Fetched mshadow just fine...
\n====================================================================
\n$ MSHADOW_USE_CUDA=0 ./build.sh blas=1
\ng++ -c -Wall -g -O3 -msse3 -Wno-unknown-pragmas -funroll-loops -I./mshadow/ -I/usr/lib -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_CBLAS=1 -o cxxnet_data.o cxxnet/io/cxxnet_data.cpp
In file included from cxxnet/io/cxxnet_data.cpp:6:
In file included from ./mshadow/mshadow/tensor.h:11:
./mshadow/mshadow/tensor_base.h:83:12: fatal error: 'cublas.h' file not found
#include <cublas.h>
^
1 error generated.
make: *** [cxxnet_data.o] Error 1
Yes, I'm using it on 10.10 with cuda. Where are you stuck at?
\n\n[quote=clustifier;61783]
\nHas someone used CXXNET on mac osx successfully?
\n(with or without cuda)
\n[/quote]
", "votes": null, "replies": [] }, { "id": 61793, "author_name": "nissanpow", "author_url": "", "post_date": "12/23/2014 23:24:25", "content": "cublas.h is located at /usr/local/cuda/include/, which is a symlink to /Developer/NVIDIA/CUDA-6.5/include for me.
\nAdd -I/usr/local/cuda/include/ to your Makefile.
\n\n\n[quote=Mark Conway;61786]
\n[quote=clustifier;61783]
\nHas someone used CXXNET on mac osx successfully?
\n(with or without cuda)
\n[/quote]
\nNope, trying though... can't seem to get by the includes. The file cblas.h is actually part of the Accelerate framework on the Mac, but now we are trying to find cublas.h. Fetched mshadow just fine...
\n====================================================================
\n$ MSHADOW_USE_CUDA=0 ./build.sh blas=1
\ng++ -c -Wall -g -O3 -msse3 -Wno-unknown-pragmas -funroll-loops -I./mshadow/ -I/usr/lib -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_CBLAS=1 -o cxxnet_data.o cxxnet/io/cxxnet_data.cpp
In file included from cxxnet/io/cxxnet_data.cpp:6:
In file included from ./mshadow/mshadow/tensor.h:11:
./mshadow/mshadow/tensor_base.h:83:12: fatal error: 'cublas.h' file not found
#include <cublas.h>
^
1 error generated.
make: *** [cxxnet_data.o] Error 1
[/quote]
", "votes": null, "replies": [] }, { "id": 61828, "author_name": "luyitian", "author_url": "", "post_date": "12/24/2014 14:51:04", "content": "Hi, anyone here uses MacOS and successfully install CXXNET? I tried to but met tons of ERRORs T.T
\n------------------------
\nFirst I use 'brew install' to get opencv and installed CUDA toolbox. Then I run ' sudo ./build.sh blas=1' and failed with 'cannot found cblas.h'. I googled cblas and found it is included in Mac's Accelerate framework. So I naively thought replacing cblas.h with Accelerate.h will solve this problem.
\n#if MSHADOW_USE_CBLAS
#include Accelerate/Accelerate.h
------------------------
\nThen I met following error:
\n/usr/include/dispatch/object.h(143): error: expected an identifier/usr/include/dispatch/object.h(362): error: identifier "dispatch_block_t" is undefined/System/Library/Frameworks/CoreFoundation.framework/Headers/CFBundle.h(148): error: format argument does not have string type
I solved this under the guide in :http://stackoverflow.com/questions/26527077/compiling-with-accelerate-framework-on-osx-yosemite
\n------------------------
\nThen this error occurs: 'ld: library not found for -lcudart' this is easy to solve, just edit the -I and -L args. on my macbook it is:
\n-I/Developer/NVIDIA/CUDA-6.5/include/ -L/Developer/NVIDIA/CUDA-6.5/lib
\n------------------------
\nThen I got this error:
\nUndefined symbols for architecture x86_64:
"std::string::_Rep::_M_destroy(std::allocator<char> const&)", referenced from:
cxxnet::SGHMCUpdater<mshadow::gpu, 1>::SGHMCUpdater(mshadow::Random<mshadow::gpu>&, mshadow::Tensor<mshadow::gpu, 1>&, mshadow::Tensor<mshadow::gpu, 1>&, char const*) in cxxnet_nnet_gpu.o
......
std::vector<cxxnet::ILayer*, std::allocator<cxxnet::ILayer*> >::_M_insert_aux(__gnu_cxx::__normal_iterator<cxxnet::ILayer**, std::vector<cxxnet::ILayer*, std::allocator<cxxnet::ILayer*> > >, cxxnet::ILayer* const&) in cxxnet_nnet_gpu.o
std::vector<cxxnet::Node<mshadow::gpu>, std::allocator<cxxnet::Node<mshadow::gpu> > >::_M_insert_aux(__gnu_cxx::__normal_iterator<cxxnet::Node<mshadow::gpu>*, std::vector<cxxnet::Node<mshadow::gpu>, std::allocator<cxxnet::Node<mshadow::gpu> > > >, cxxnet::Node<mshadow::gpu> const&) in cxxnet_nnet_gpu.o
...
ld: symbol(s) not found for architecture x86_64
I googled it and found this may due to the update of xcode from 5.x to 6.0(I use the newest xcode). And I have no idea how to fix it.
\n------------------------
\nFinally I give up using cblas and turn to mkl in https://software.intel.com/en-us/intel-parallel-studio-xe. Currently I am downloading the 30days free trial for XE (does XE have a free version for student? I could not find any link to it)
\n------------------------
\nupdate: I installed OpenBLAS and add '-I/opt/OpenBLAS/include' in makefile but still got an 'ld: symbol(s) not found for architecture x86_64' error. it seems CXXNET just cannot run on newest MacOS/xcode. Now I have give up and found alternatives.
", "votes": null, "replies": [] }, { "id": 61830, "author_name": "mconway", "author_url": "", "post_date": "12/24/2014 15:15:35", "content": "I took the same route you did. I installed Intel Parallel Studio to get MKL, then set up all the include paths, and was able to at least compile some of the objects. But then the make was trying to compile with "nvcc" to create the cxxnet executable and that died a horrible death. I gave up after a few hours... it was like hacking through the jungle with a machete. Best of luck!
", "votes": null, "replies": [] }, { "id": 61848, "author_name": "tudinhnguyen", "author_url": "", "post_date": "12/25/2014 01:25:02", "content": "[quote=Bing Xu;60426]
\nHi all,
\nI just made a guide of how to use conv net to achieve NLL of 1.382285. On a modern GPU, it only takes 5 minute to get this result.
\nPlease check https://github.com/antinucleon/cxxnet/tree/master/example/kaggle_bowl , and don't forget to star CXXNET :)
\nCXXNET is yet another neural network toolkit. It is able to run on NVIDIA GPU or CPU. If you want to run on CPU, you only need OpenCV & MKL(or OpenBlas/ATLAS) installed. No other dependence is required.
\nPS. I made it in a hurry, so if there is any problem please let me know!
\n[/quote]
\nMay I ask about the training NLL and the distribution of predicted labels for test data?
", "votes": null, "replies": [] }, { "id": 61849, "author_name": "tudinhnguyen", "author_url": "", "post_date": "12/25/2014 01:29:09", "content": "[quote=Bing Xu;60426]
\nHi all,
\nI just made a guide of how to use conv net to achieve NLL of 1.382285. On a modern GPU, it only takes 5 minute to get this result.
\nPlease check https://github.com/antinucleon/cxxnet/tree/master/example/kaggle_bowl , and don't forget to star CXXNET :)
\nCXXNET is yet another neural network toolkit. It is able to run on NVIDIA GPU or CPU. If you want to run on CPU, you only need OpenCV & MKL(or OpenBlas/ATLAS) installed. No other dependence is required.
\nPS. I made it in a hurry, so if there is any problem please let me know!
\n[/quote]
\nMay I ask about the training NLL and the distribution of predicted labels for test data?
", "votes": null, "replies": [] }, { "id": 61887, "author_name": "mconway", "author_url": "", "post_date": "12/26/2014 02:20:38", "content": "Hi Bing:
\nRunning on Mac Yosemite, got everything built, converted the images, but now getting the following error:
\nError:Assignment: Shape of Tensors in expression is not consistent with target
\nlocalhost:kaggle_bowl$ ../../bin/cxxnet bowl.conf
CXXNetTrainer, devCPU=1
ConvolutionLayer: nstep=256
ConvolutionLayer: nstep=256
ConvolutionLayer: nstep=128
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
node[0].shape: 256,3,48,48
node[1].shape: 256,96,12,12
node[2].shape: 256,96,12,12
node[3].shape: 256,96,6,6
node[4].shape: 256,128,8,8
node[5].shape: 256,128,8,8
node[6].shape: 256,128,8,8
node[7].shape: 256,128,8,8
node[8].shape: 256,128,4,4
node[9].shape: 1,1,256,2048
node[10].shape: 1,1,256,512
node[11].shape: 1,1,256,512
node[12].shape: 1,1,256,512
node[13].shape: 1,1,256,512
node[14].shape: 1,1,256,121
ThreadImagePageIterator:image_list=./train.lst, bin=./train.bin
cannot find models/image_mean.bin: create mean image, this will take some time...
Error:Assignment: Shape of Tensors in expression is not consistent with target
I have couple of questions with cxxnet/convert tools. First, why "convert -resize 48x48\\!" has exclamation mark? I can't find anything in convert/ImageMagik documentation. If I remove it I got similar error as Mark Conway posted, if I keep it I don't have this issue. Second question, when I run cxxnet with default configuration, it exit with the following:
\nThreadImagePageIterator:image_list=./train.lst, bin=./train.bin
cannot find models/image_mean.bin: create mean image, this will take some time...
[ 30000] images processed, 1 sec elapsedsave mean image to models/image_mean.bin..
ThreadBufferIterator: buffer_size=2
ThreadImagePageIterator:image_list=./train.lst, bin=./train.bin
loading mean image from models/image_mean.bin
ThreadBufferIterator: buffer_size=2
initializing end, start working
update round 0
and only generates models/0000.model file and models/image_mean.bin? The exit status I get (I printed via echo $?) is 1. No other messages. I don't know what's going wrong, do I have insufficient RAM or program is crashed? The code uses Quadro NVS 295 GPU.
\nThanks,
\nValentin
", "votes": null, "replies": [] }, { "id": 61919, "author_name": "nissanpow", "author_url": "", "post_date": "12/26/2014 23:01:27", "content": "The exclamation mark means ignore aspect ratio:
\n[quote]
\nIgnore Aspect Ratio ('!' flag)
If you want you can force "-resize" to ignore the aspect ratio and distort the image so it always generates an image exactly the size specified. This is done by adding the character '!' to the size. Unfortunately this character is also sometimes used for special purposes by various UNIX command line shells. So you may have to escape the character somehow to preserve it.
[/quote]
\nReference: http://www.imagemagick.org/Usage/resize/
\n\n[quote=vk.net;61917]
\nwhy "convert -resize 48x48\\!" has exclamation mark?
\n[/quote]
", "votes": null, "replies": [] }, { "id": 61929, "author_name": "billyinn", "author_url": "", "post_date": "12/27/2014 13:12:12", "content": "I've written a summary of the installation of cxxnet on CentOS via blas, based on the error I've encountered during the installation.
\nHere is the summary: https://github.com/billy-inn/cxxnet/tree/master/Install
\nHope it may help.
", "votes": null, "replies": [] }, { "id": 61934, "author_name": "mconway", "author_url": "", "post_date": "12/27/2014 15:26:39", "content": "Hi Valentin:
\nI'm getting pretty close. I believe the "Shape of Tensors not consistent" error is related to the image list processed by cxxnet, as you point out. So, I will experiment with the convert options and see if we can finish this last step.
\nRegards,
\nMark
\n[quote=vk.net;61917]
\nI have couple of questions with cxxnet/convert tools. First, why "convert -resize 48x48\\!" has exclamation mark? I can't find anything in convert/ImageMagik documentation. If I remove it I got similar error as Mark Conway posted, if I keep it I don't have this issue. Second question, when I run cxxnet with default configuration, it exit with the following:
\nThreadImagePageIterator:image_list=./train.lst, bin=./train.bin
cannot find models/image_mean.bin: create mean image, this will take some time...
[ 30000] images processed, 1 sec elapsedsave mean image to models/image_mean.bin..
ThreadBufferIterator: buffer_size=2
ThreadImagePageIterator:image_list=./train.lst, bin=./train.bin
loading mean image from models/image_mean.bin
ThreadBufferIterator: buffer_size=2
initializing end, start working
update round 0
and only generates models/0000.model file and models/image_mean.bin? The exit status I get (I printed via echo $?) is 1. No other messages. I don't know what's going wrong, do I have insufficient RAM or program is crashed? The code uses Quadro NVS 295 GPU.
\nThanks,
\nValentin
\n[/quote]
", "votes": null, "replies": [] }, { "id": 62072, "author_name": "nanochef", "author_url": "", "post_date": "12/30/2014 00:17:00", "content": "Nissan, can you elaborate where to add -I/usr/local/cuda/include in the Makefile? I'm a noob so it isn't obvious. :)
\n[quote=Nissan Pow;61793]
\ncublas.h is located at /usr/local/cuda/include/, which is a symlink to /Developer/NVIDIA/CUDA-6.5/include for me.
\nAdd -I/usr/local/cuda/include/ to your Makefile.
\n[quote=Mark Conway;61786]
\n[quote=clustifier;61783]
\nHas someone used CXXNET on mac osx successfully?
\n(with or without cuda)
\n[/quote]
\nNope, trying though... can't seem to get by the includes. The file cblas.h is actually part of the Accelerate framework on the Mac, but now we are trying to find cublas.h. Fetched mshadow just fine...
\n====================================================================
\n$ MSHADOW_USE_CUDA=0 ./build.sh blas=1
\ng++ -c -Wall -g -O3 -msse3 -Wno-unknown-pragmas -funroll-loops -I./mshadow/ -I/usr/lib -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_CBLAS=1 -o cxxnet_data.o cxxnet/io/cxxnet_data.cpp
In file included from cxxnet/io/cxxnet_data.cpp:6:
In file included from ./mshadow/mshadow/tensor.h:11:
./mshadow/mshadow/tensor_base.h:83:12: fatal error: 'cublas.h' file not found
#include <cublas.h>
^
1 error generated.
make: *** [cxxnet_data.o] Error 1
[/quote]
\n[/quote]
", "votes": null, "replies": [] }, { "id": 62073, "author_name": "vkuznet", "author_url": "", "post_date": "12/30/2014 00:45:14", "content": "Here the part of Makefile I modified:
\nexport CFLAGS = -Wall -g -O3 -msse3 -Wno-unknown-pragmas -funroll-loops -I./mshadow/
CFLAGS+=-I/opt/cuda/include
ifeq ($(blas),1)
LDFLAGS= -L/opt/cuda/lib -lm -lcudart -lcublas -lcurand -lz `pkg-config --libs opencv` -lcblas -lgfortran -lblas
CFLAGS+= -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_CBLAS=1
else
LDFLAGS= -lm -lcudart -lcublas -lmkl_core -lmkl_intel_lp64 -lmkl_intel_thread -liomp5 -lpthread -lcurand -lz `pkg-config --libs opencv`
endif
Basically you add additional CFLAGS with you CUDA include path and modiy LDFLAGS as well with path to CUDA libraries. The CUDA toolkit (https://developer.nvidia.com/cuda-toolkit) comes from NVidia web site and it includes cublas.h
", "votes": null, "replies": [] }, { "id": 62083, "author_name": "esorar", "author_url": "", "post_date": "12/30/2014 04:34:15", "content": "Thanks Bing for this eye opening example by running it with GPU Cuda cores.
\nAs much as I can follow you use one of Linux flavor to utilize GPU and maybe some other people has success on MacOS. I can run your examples on CPU on Ubuntu VM with vmplayer on a Win8.1 host however I wonder whether I can utilize host Nvidia GPU tied to host Win8.1. Is there Anybody managed to run this on Win8.1 OS with GPU? There is a MinGW solution: http://pavel.surmenok.com/2014/05/31/installing-theano-with-gpu-on-windows-64-bit/
\nbut not yet tried maybe somebody can point a better, like cygwin possibility?
\nThanks
", "votes": null, "replies": [] }, { "id": 62150, "author_name": "jiweiliu", "author_url": "", "post_date": "12/31/2014 21:30:13", "content": "Hi everyone, finally I installed cxxnet on my laptop, which has a low-end Geforce GT 630m graphic card. The score is 1.42 and the training time is one hour. But I'm still happy it works. Thank you, Bing! :D
", "votes": null, "replies": [] }, { "id": 62194, "author_name": "senecaur", "author_url": "", "post_date": "01/01/2015 23:43:31", "content": "I am using a CNN approach using Caffe rather than cxxnet. My network structure is very similar to what is defined in Bing's shared code. Out of interest, can anyone tell me how many training iterations the cxxnet code runs for? I can't seem to see where that is defined in the code.
", "votes": null, "replies": [] }, { "id": 62204, "author_name": "gregoirel", "author_url": "", "post_date": "01/02/2015 07:29:46", "content": "[quote=Senecaur;62194]
\nI am using a CNN approach using Caffe rather than cxxnet. My network structure is very similar to what is defined in Bing's shared code. Out of interest, can anyone tell me how many training iterations the cxxnet code runs for? I can't seem to see where that is defined in the code.
\n[/quote]
\n\n45 in Bing's example. The entries in the .conf file are "max_round" and "num_round".
\n\nG
", "votes": null, "replies": [] }, { "id": 62214, "author_name": "abhishek", "author_url": "", "post_date": "01/02/2015 14:56:46", "content": "[quote=gregl;61632]
\n[quote=clustifier;61601]
\nthanks.
\nbut why our md5's are different?
\n[/quote]
\nI built im2bin with blas, not the default mlk.
\nAlso, upstream, the md5sum of train.lst is:
\n399ab27e818cabcb5bd2158c15451a80 train.lst
\nwith the first 5 lines:
\n3406 10 data/train/chaetognath_non_sagitta/23677.jpg
22212 90 data/train/radiolarian_chain/157296.jpg
19772 83 data/train/protist_fuzzy_olive/147898.jpg
23435 98 data/train/siphonophore_calycophoran_rocketship_young/36262.jpg
18710 72 data/train/hydromedusae_solmaris/47760.jpg
[/quote]
\nhow did you build im2bin with blas when mshadow needs mkl?
\n\nEDIT: Nevermind
", "votes": null, "replies": [] }, { "id": 62236, "author_name": "", "author_url": "", "post_date": "01/03/2015 17:39:20", "content": "Hi Valentin,
\nin case you're still having the issue you described in your second question, I had exactly the same with a GTX 680 GPU.
\nI discovered a warning during the compilation of 'tensor_gpu-inl.cuh' which is part of mshadow and located in 'cxxnet-master/mshadow/mshadow/cuda'. The warning said that the CUDA architecture of the GPU could not be determined and will be set to 2.0 automatically. It seemed that the __CUDA_ARCH__ macro, that is checked for the architecture, was broken. Setting this macro by hand like
\n#define __CUDA_ARCH__ 300
\nresolved the issue for me (in my case, the 300 comes from the GTX 680 having compute capability 3.0). Maybe this can help with your problem too...
\nBest,
\nMatthias
\np.s.: Thanks a lot Bing for making this available!
\n[quote=vk.net;61917]
\nSecond question, when I run cxxnet with default configuration, it exit with the following:
\nThreadImagePageIterator:image_list=./train.lst, bin=./train.bin
cannot find models/image_mean.bin: create mean image, this will take some time...
[ 30000] images processed, 1 sec elapsedsave mean image to models/image_mean.bin..
ThreadBufferIterator: buffer_size=2
ThreadImagePageIterator:image_list=./train.lst, bin=./train.bin
loading mean image from models/image_mean.bin
ThreadBufferIterator: buffer_size=2
initializing end, start working
update round 0
and only generates models/0000.model file and models/image_mean.bin? The exit status I get (I printed via echo $?) is 1. No other messages. I don't know what's going wrong, do I have insufficient RAM or program is crashed? The code uses Quadro NVS 295 GPU.
\nThanks,
\nValentin
\n[/quote]
", "votes": null, "replies": [] }, { "id": 62285, "author_name": "telser", "author_url": "", "post_date": "01/05/2015 03:08:59", "content": "[quote=vk.net;62073]
\nHere the part of Makefile I modified:
\nexport CFLAGS = -Wall -g -O3 -msse3 -Wno-unknown-pragmas -funroll-loops -I./mshadow/
CFLAGS+=-I/opt/cuda/include
ifeq ($(blas),1)
LDFLAGS= -L/opt/cuda/lib -lm -lcudart -lcublas -lcurand -lz `pkg-config --libs opencv` -lcblas -lgfortran -lblas
CFLAGS+= -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_CBLAS=1
else
LDFLAGS= -lm -lcudart -lcublas -lmkl_core -lmkl_intel_lp64 -lmkl_intel_thread -liomp5 -lpthread -lcurand -lz `pkg-config --libs opencv`
endif
Basically you add additional CFLAGS with you CUDA include path and modiy LDFLAGS as well with path to CUDA libraries. The CUDA toolkit (https://developer.nvidia.com/cuda-toolkit) comes from NVidia web site and it includes cublas.h
\n[/quote]
\n\nThanks for the advice; this resolved the cublas issue. I was wondering if it worked for you with just that, though; after making the modification, I now get
\nsh build.sh blas=1
\n
Fetch mshadow...
fatal: destination path 'mshadow' already exists and is not an empty directory.
nvcc -c -o cxxnet_nnet_gpu.o --use_fast_math -g -O3 -ccbin g++ -Xcompiler "-Wall -g -O3 -msse3 -Wno-unknown-pragmas -funroll-loops -I./mshadow/ -I/usr/local/cuda/include -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_CBLAS=1" cxxnet/nnet/cxxnet_nnet.cu
/bin/sh: 1: nvcc: not found
make: *** [cxxnet_nnet_gpu.o] Error 127
Torgos, looks like you havn't added CUDA to your path?? i.e. something like:
\nexport PATH=$PATH:/usr/local/cuda-6.5/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-6.5/lib64:/usr/local/cuda-6.5/include
Right you are; I thought I had added it to my bashrc. Thanks!
\nEdit.Unfortunately still seems to be broken, though. Blargh. Has anyone gotten this installed on a Ubuntu 14.04 on an Amazon GPU instance?
\nEdit 2: I think I got it; the ubuntu distribution is 64 bit, so you will get compatibility issues with the 32 bit library; point the makefile at -L/usr/local/cuda/lib64 instead. You also need to add the -pthread to
\nCFLAGS+= -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_CBLAS=1 -pthread
\nas per
\nhttps://github.com/antinucleon/cxxnet/issues/14
\nNo errors (other than 'mshadow already exists') when running sh build.sh blas=1 after that. Still need to figure out what paths to change the Makefile for im2bin, though.
\nEdit 3. Modify the first few lines in the im2bin Makefile to
\nexport CFLAGS = -Wall -O3 -msse3 -Wno-unknown-pragmas -funroll-loops -I../mshadow/ -I..
\nCFLAGS+=-I/usr/local/cuda/include
export LDFLAGS=-L/usr/local/cuda/lib64 -lm -lcudart -lcublas -lcurand -lz `pkg-config --libs opencv` -lblas
export NVCCFLAGS = -g -O3 -ccbin $(CXX)
CFLAGS+= -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_CBLAS=1 -pthread
It should compile correctly now; unfortunately, the actual command doesn't work! I suspect that the issue was in the argument passed when train.lst was generated.
", "votes": null, "replies": [] }, { "id": 62324, "author_name": "griesrt", "author_url": "", "post_date": "01/06/2015 02:34:55", "content": "Did anyone get it running on the Macbook on OSX?
", "votes": null, "replies": [] }, { "id": 62328, "author_name": "jfkingiii", "author_url": "", "post_date": "01/06/2015 04:55:00", "content": "I stumbled across this on the caffe github repo today:
\n"Be aware of the competing fast GPU neural network library CXXNET"
\nhttps://github.com/BVLC/caffe/issues/382
\nI am so impressed with with what Tianqi and Bing have accomplished with cxxnet and xgboost. Please, keep doing what you're doing!
", "votes": null, "replies": [] }, { "id": 62332, "author_name": "griesrt", "author_url": "", "post_date": "01/06/2015 06:50:55", "content": "Okay, I got past a few problems but now I'm running into this:
\nUndefined symbols for architecture x86_64:
"std::string::_Rep::_M_destroy(std::allocator<char> const&)", referenced from:
cxxnet::SGHMCUpdater<mshadow::gpu, 1>::SGHMCUpdater(mshadow::Random<mshadow::gpu>&, mshadow::Tensor<mshadow::gpu, 1>&, mshadow::Tensor<mshadow::gpu, 1>&, char const*) in cxxnet_nnet_gpu.o
cxxnet::SGHMCUpdater<mshadow::gpu, 1>::~SGHMCUpdater() in cxxnet_nnet_gpu.o
cxxnet::SGDUpdater<mshadow::gpu, 1>::SGDUpdater(mshadow::Tensor<mshadow::gpu, 1>&, mshadow::Tensor<mshadow::gpu, 1>&, char const*) in cxxnet_nnet_gpu.o
cxxnet::SGDUpdater<mshadow::gpu, 1>::~SGDUpdater() in cxxnet_nnet_gpu.o
cxxnet::SGDUpdater<mshadow::gpu, 1>::~SGDUpdater() in cxxnet_nnet_gpu.o
cxxnet::SGHMCUpdater<mshadow::gpu, 3>::SGHMCUpdater(mshadow::Random<mshadow::gpu>&, mshadow::Tensor<mshadow::gpu, 3>&, mshadow::Tensor<mshadow::gpu, 3>&, char const*) in cxxnet_nnet_gpu.o
cxxnet::SGHMCUpdater<mshadow::gpu, 3>::~SGHMCUpdater() in cxxnet_nnet_gpu.o
...
"std::string::_Rep::_S_empty_rep_storage", referenced from:
cxxnet::SGHMCUpdater<mshadow::gpu, 1>::SGHMCUpdater(mshadow::Random<mshadow::gpu>&, mshadow::Tensor<mshadow::gpu, 1>&, mshadow::Tensor<mshadow::gpu, 1>&, char const*) in cxxnet_nnet_gpu.o
cxxnet::SGHMCUpdater<mshadow::gpu, 1>::~SGHMCUpdater() in cxxnet_nnet_gpu.o
cxxnet::SGDUpdater<mshadow::gpu, 1>::SGDUpdater(mshadow::Tensor<mshadow::gpu, 1>&, mshadow::Tensor<mshadow::gpu, 1>&, char const*) in cxxnet_nnet_gpu.o
cxxnet::SGDUpdater<mshadow::gpu, 1>::~SGDUpdater() in cxxnet_nnet_gpu.o
cxxnet::SGDUpdater<mshadow::gpu, 1>::~SGDUpdater() in cxxnet_nnet_gpu.o
cxxnet::SGHMCUpdater<mshadow::gpu, 3>::SGHMCUpdater(mshadow::Random<mshadow::gpu>&, mshadow::Tensor<mshadow::gpu, 3>&, mshadow::Tensor<mshadow::gpu, 3>&, char const*) in cxxnet_nnet_gpu.o
cxxnet::SGHMCUpdater<mshadow::gpu, 3>::~SGHMCUpdater() in cxxnet_nnet_gpu.o
...
"std::string::assign(char const*, unsigned long)", referenced from:
cxxnet::SGHMCUpdater<mshadow::gpu, 1>::SGHMCUpdater(mshadow::Random<mshadow::gpu>&, mshadow::Tensor<mshadow::gpu, 1>&, mshadow::Tensor<mshadow::gpu, 1>&, char const*) in cxxnet_nnet_gpu.o
cxxnet::SGDUpdater<mshadow::gpu, 1>::SGDUpdater(mshadow::Tensor<mshadow::gpu, 1>&, mshadow::Tensor<mshadow::gpu, 1>&, char const*) in cxxnet_nnet_gpu.o
cxxnet::SGHMCUpdater<mshadow::gpu, 3>::SGHMCUpdater(mshadow::Random<mshadow::gpu>&, mshadow::Tensor<mshadow::gpu, 3>&, mshadow::Tensor<mshadow::gpu, 3>&, char const*) in cxxnet_nnet_gpu.o
cxxnet::SGDUpdater<mshadow::gpu, 3>::SGDUpdater(mshadow::Tensor<mshadow::gpu, 3>&, mshadow::Tensor<mshadow::gpu, 3>&, char const*) in cxxnet_nnet_gpu.o
cxxnet::SGHMCUpdater<mshadow::gpu, 2>::SGHMCUpdater(mshadow::Random<mshadow::gpu>&, mshadow::Tensor<mshadow::gpu, 2>&, mshadow::Tensor<mshadow::gpu, 2>&, char const*) in cxxnet_nnet_gpu.o
cxxnet::SGDUpdater<mshadow::gpu, 2>::SGDUpdater(mshadow::Tensor<mshadow::gpu, 2>&, mshadow::Tensor<mshadow::gpu, 2>&, char const*) in cxxnet_nnet_gpu.o
cxxnet::utils::MetricSet::AddMetric(char const*) in cxxnet_nnet_gpu.o
...
"std::string::assign(std::string const&)", referenced from:
std::vector<std::pair<std::string, std::string>, std::allocator<std::pair<std::string, std::string> > >::_M_insert_aux(__gnu_cxx::__normal_iterator<std::pair<std::string, std::string>*, std::vector<std::pair<std::string, std::string>, std::allocator<std::pair<std::string, std::string> > > >, std::pair<std::string, std::string> const&) in cxxnet_nnet_gpu.o
"std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, std::allocator<char> const&)", referenced from:
cxxnet::NetConfigHelper::SetParam(char const*, char const*) in cxxnet_nnet_gpu.o
"std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(std::string const&)", referenced from:
cxxnet::NetConfigHelper::SetParam(char const*, char const*) in cxxnet_nnet_gpu.o
std::vector<std::pair<std::string, std::string>, std::allocator<std::pair<std::string, std::string> > >::_M_insert_aux(__gnu_cxx::__normal_iterator<std::pair<std::string, std::string>*, std::vector<std::pair<std::string, std::string>, std::allocator<std::pair<std::string, std::string> > > >, std::pair<std::string, std::string> const&) in cxxnet_nnet_gpu.o
"std::__throw_length_error(char const*)", referenced from:
std::vector<float, std::allocator<float> >::_M_fill_insert(__gnu_cxx::__normal_iterator<float*, std::vector<float, std::allocator<float> > >, unsigned long, float const&) in cxxnet_nnet_gpu.o
std::vector<std::vector<float, std::allocator<float> >, std::allocator<std::vector<float, std::allocator<float> > > >::_M_fill_insert(__gnu_cxx::__normal_iterator<std::vector<float, std::allocator<float> >*, std::vector<std::vector<float, std::allocator<float> >, std::allocator<std::vector<float, std::allocator<float> > > > >, unsigned long, std::vector<float, std::allocator<float> > const&) in cxxnet_nnet_gpu.o
std::vector<float, std::allocator<float> >::_M_insert_aux(__gnu_cxx::__normal_iterator<float*, std::vector<float, std::allocator<float> > >, float const&) in cxxnet_nnet_gpu.o
std::vector<int, std::allocator<int> >::_M_fill_insert(__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int> > >, unsigned long, int const&) in cxxnet_nnet_gpu.o
std::vector<cxxnet::IUpdater*, std::allocator<cxxnet::IUpdater*> >::_M_insert_aux(__gnu_cxx::__normal_iterator<cxxnet::IUpdater**, std::vector<cxxnet::IUpdater*, std::allocator<cxxnet::IUpdater*> > >, cxxnet::IUpdater* const&) in cxxnet_nnet_gpu.o
std::vector<cxxnet::ILayer*, std::allocator<cxxnet::ILayer*> >::_M_insert_aux(__gnu_cxx::__normal_iterator<cxxnet::ILayer**, std::vector<cxxnet::ILayer*, std::allocator<cxxnet::ILayer*> > >, cxxnet::ILayer* const&) in cxxnet_nnet_gpu.o
std::vector<cxxnet::Node<mshadow::gpu>, std::allocator<cxxnet::Node<mshadow::gpu> > >::_M_insert_aux(__gnu_cxx::__normal_iterator<cxxnet::Node<mshadow::gpu>*, std::vector<cxxnet::Node<mshadow::gpu>, std::allocator<cxxnet::Node<mshadow::gpu> > > >, cxxnet::Node<mshadow::gpu> const&) in cxxnet_nnet_gpu.o
...
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [bin/cxxnet] Error 1
[quote=Torgos;62302]
\nEdit 3. Modify the first few lines in the im2bin Makefile to
\nexport CFLAGS = -Wall -O3 -msse3 -Wno-unknown-pragmas -funroll-loops -I../mshadow/ -I..
\nCFLAGS+=-I/usr/local/cuda/include
export LDFLAGS=-L/usr/local/cuda/lib64 -lm -lcudart -lcublas -lcurand -lz `pkg-config --libs opencv` -lblas
export NVCCFLAGS = -g -O3 -ccbin $(CXX)
CFLAGS+= -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_CBLAS=1 -pthread
It should compile correctly now; unfortunately, the actual command doesn't work! I suspect that the issue was in the argument passed when train.lst was generated.
\n[/quote]
This bit worked for me. Thanks.
At first the command didn't work for me either and I was wondering why. Turns out there was a permissions issue cos I compiled the im2bin on an mounted NTFS partition instead of EXT. Ended up moving the executable to the EXT partition, gave chmod +x and command ran smoothly.
To Matthias: indeed you found a remedy. I applied macro definition and recompiled the code. Now it runs!!! So, for those who experience silent crash, just follow Matthias advice and define __CUDA_ARCH__ in cxxnet/mshadow/mshadow/cuda/tensor_gpu-inl.cuh file.
", "votes": null, "replies": [] }, { "id": 62347, "author_name": "abhishek", "author_url": "", "post_date": "01/06/2015 13:00:13", "content": "is it possible to compile CXXNET without OpenCV?
", "votes": null, "replies": [] }, { "id": 62378, "author_name": "griesrt", "author_url": "", "post_date": "01/06/2015 20:28:32", "content": "I'm trying to get this running on a Macbook. What version of OpenCV is everyone using? I have version 2.4.9 and I'm getting this error:
\n\nUndefined symbols for architecture x86_64:
"cv::imread(std::string const&, int)", referenced from:
cxxnet::ImageIterator::LoadImage(mshadow::TensorContainer<mshadow::cpu, 3>&, cxxnet::DataInst&, char const*) in cxxnet_data.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [bin/cxxnet] Error 1
Any ideas?
", "votes": null, "replies": [] }, { "id": 62410, "author_name": "vkuznet", "author_url": "", "post_date": "01/07/2015 17:22:19", "content": "Hi, when I run the code, it produces the output with train-error values for each round. Can someone explain how they're defined.
", "votes": null, "replies": [] }, { "id": 62509, "author_name": "sneogi", "author_url": "", "post_date": "01/09/2015 16:23:15", "content": "Hello,
\nI finally got the code compiled and linked, although a cpu version, because I have no GPU at the moment. I got the training done and found the 45 models in the models directory as expected along with the image means.
\n$ ../../bin/cxxnet bowl_48x48.conf
CXXNetTrainer, devCPU=1
ConvolutionLayer: nstep=256
.
[output removed]
\n.
\nThreadImagePageIterator:image_list=./train_48x48.lst, bin=./train_48x48.bin
cannot find models/image_mean_48x48.bin: create mean image, this will take some time...
[ 30000] images processed, 1 sec elapsedsave mean image to models/image_mean_48x48.bin..
ThreadBufferIterator: buffer_size=2
ThreadImagePageIterator:image_list=./train_48x48.lst, bin=./train_48x48.bin
loading mean image from models/image_mean_48x48.bin
ThreadBufferIterator: buffer_size=2
initializing end, start working
round 0:[ 100] 246 sec elapsed[1] train-error:0.797901
.
[output removed]
\n.
round 44:[ 100] 20950 sec elapsed[45] train-error:0.352688
updating end, 21135 sec in all
\nNext, I invoked the predictor. It continues for a while and then halts with an error Error:decoding fail as seen in the trace below. I have the output txt file, where I see 67400 images were processed. Inspecting the code, I found that error message in the file cxxnet_iter_thread_imbin-inl.hpp, line 65, function inline static void LoadImage().
\n$ ../../bin/cxxnet pred_48x48.conf
CXXNetTrainer, devCPU=1
ConvolutionLayer: nstep=100
ConvolutionLayer: nstep=100
ConvolutionLayer: nstep=100
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
node[0].shape: 100,3,48,48
node[1].shape: 100,96,12,12
node[2].shape: 100,96,12,12
node[3].shape: 100,96,6,6
node[4].shape: 100,128,8,8
node[5].shape: 100,128,8,8
node[6].shape: 100,128,8,8
node[7].shape: 100,128,8,8
node[8].shape: 100,128,4,4
node[9].shape: 1,1,100,2048
node[10].shape: 1,1,100,512
node[11].shape: 1,1,100,512
node[12].shape: 1,1,100,512
node[13].shape: 1,1,100,512
node[14].shape: 1,1,100,121
ThreadImagePageIterator:image_list=./train_48x48.lst, bin=./train_48x48.bin
loading mean image from models/image_mean_48x48.bin
ThreadBufferIterator: buffer_size=2
ThreadImagePageIterator:image_list=./test_48x48.lst, bin=./test_48x48.bin
loading mean image from models/image_mean_48x48.bin
ThreadBufferIterator: buffer_size=2
initializing end, start working
start predicting...
Error:decoding fail
I am suspecting something went wrong during the conversion and subsequently during loading of the images. Did anyone come across this situation? Any hints?
\nThanks.
\n\nEdit: Tried out again. It works.
", "votes": null, "replies": [] }, { "id": 62815, "author_name": "exored", "author_url": "", "post_date": "01/17/2015 06:15:31", "content": "Hi Mark, Can you give a brief description on how you managed to build on mac? Thank you.
\n[quote=Mark Conway;61887]
\nHi Bing:
\nRunning on Mac Yosemite, got everything built, converted the images, but now getting the following error:
\nError:Assignment: Shape of Tensors in expression is not consistent with target
\nlocalhost:kaggle_bowl$ ../../bin/cxxnet bowl.conf
CXXNetTrainer, devCPU=1
ConvolutionLayer: nstep=256
ConvolutionLayer: nstep=256
ConvolutionLayer: nstep=128
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
SGDUpdater: eta=0.010000, mom=0.900000
SGDUpdater: eta=0.020000, mom=0.900000
node[0].shape: 256,3,48,48
node[1].shape: 256,96,12,12
node[2].shape: 256,96,12,12
node[3].shape: 256,96,6,6
node[4].shape: 256,128,8,8
node[5].shape: 256,128,8,8
node[6].shape: 256,128,8,8
node[7].shape: 256,128,8,8
node[8].shape: 256,128,4,4
node[9].shape: 1,1,256,2048
node[10].shape: 1,1,256,512
node[11].shape: 1,1,256,512
node[12].shape: 1,1,256,512
node[13].shape: 1,1,256,512
node[14].shape: 1,1,256,121
ThreadImagePageIterator:image_list=./train.lst, bin=./train.bin
cannot find models/image_mean.bin: create mean image, this will take some time...
Error:Assignment: Shape of Tensors in expression is not consistent with target
[/quote]
", "votes": null, "replies": [] }, { "id": 62917, "author_name": "ashhafez", "author_url": "", "post_date": "01/19/2015 23:03:27", "content": "Just in case it's of help to anyone is still having the below opencv x86_64 error on Mac with the latest XCode, I managed to get it compiling on mine by re-compiling opencv to use -stdlib=libstdc++ instead of the default -stdlib=libc++, via the shortcut of re-installing it with the following command:
\n"brew reinstall opencv --with-cuda"
\n(if you do a "brew edit opencv" you'll notice it's turning on -stdlib=libstdc++ when the cuda option is set). you of course need to have the cuda stuff installed as well
\nI built cxxnet with blas=1 and had these settings in the Makefile (though perhaps not all are needed, since I was messing around with it for a while)
\nLDFLAGS= -lm -lcudart -lcublas -lcurand -lz `pkg-config --libs opencv` `pkg-config --libs openblas` -lblas -L/usr/local/cuda/lib
CFLAGS+= -stdlib=libstdc++ -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_CBLAS=1 -DMSHADOW_USE_CUDA=1 -I/usr/local/Cellar/openblas/0.2.13/include -I/usr/local/cuda/include
-----
\nUndefined symbols for architecture x86_64:
"cv::imread(std::string const&, int)", referenced from:
-----
", "votes": null, "replies": [] }, { "id": 62936, "author_name": "griesrt", "author_url": "", "post_date": "01/20/2015 19:25:41", "content": "Wow. Thank you so much Ash. I've been blocked by this error for quite some time.
", "votes": null, "replies": [] }, { "id": 63180, "author_name": "obaiev", "author_url": "", "post_date": "01/26/2015 08:02:06", "content": "Hi Bing,
\nThank you for sharing the guide.
\n
In the config file you set image shape to (3,48,48). As I can understand "3" is a number of input channels. But images in dataset is in gray-scale and have only one channel. How you convert from 1 to 3 channels? Is it just a copy?
I use opencv to decode, opencv's output for jpg is 3 channel, and all are same.
\n[quote=Oleksandr Baiev;63180]
\nHi Bing,
\nThank you for sharing the guide.
\n
In the config file you set image shape to (3,48,48). As I can understand "3" is a number of input channels. But images in dataset is in gray-scale and have only one channel. How you convert from 1 to 3 channels? Is it just a copy?
[/quote]
", "votes": null, "replies": [] }, { "id": 63208, "author_name": "obaiev", "author_url": "", "post_date": "01/26/2015 20:40:41", "content": "Oh, thank you!
\n[quote=Bing Xu;63204]
\nI use opencv to decode, opencv's output for jpg is 3 channel, and all are same.
\n[quote=Oleksandr Baiev;63180]
\nHi Bing,
\nThank you for sharing the guide.
\n
In the config file you set image shape to (3,48,48). As I can understand "3" is a number of input channels. But images in dataset is in gray-scale and have only one channel. How you convert from 1 to 3 channels? Is it just a copy?
[/quote]
\n[/quote]
", "votes": null, "replies": [] }, { "id": 63220, "author_name": "jfkingiii", "author_url": "", "post_date": "01/26/2015 23:53:40", "content": "[quote=Bing Xu;63204]
\nI use opencv to decode, opencv's output for jpg is 3 channel, and all are same.
\n[/quote]
\nI believe libjpeg, which is required for openCV, will decode greyscale images into a single channel. I'm not sure if any additional overhead is introduced by processing three identical channels in the convnet.
\nCaffe appears to determine image size by reading the jpeg metadata and it understands greyscale. However I have found cxxnet to be substantially faster than caffe on this problem.
", "votes": null, "replies": [] }, { "id": 63272, "author_name": "mdjawad", "author_url": "", "post_date": "01/28/2015 12:53:48", "content": "[quote=Ertuka;62083]
\nThanks Bing for this eye opening example by running it with GPU Cuda cores.
\nAs much as I can follow you use one of Linux flavor to utilize GPU and maybe some other people has success on MacOS. I can run your examples on CPU on Ubuntu VM with vmplayer on a Win8.1 host however I wonder whether I can utilize host Nvidia GPU tied to host Win8.1. Is there Anybody managed to run this on Win8.1 OS with GPU? There is a MinGW solution: http://pavel.surmenok.com/2014/05/31/installing-theano-with-gpu-on-windows-64-bit/
\nbut not yet tried maybe somebody can point a better, like cygwin possibility?
\nThanks
\n[/quote]
\nErtuka, Could you let me know the changes you made to the build, I am running on ubuntu using VMware and I am stuck with the build issues.
", "votes": null, "replies": [] }, { "id": 63287, "author_name": "esorar", "author_url": "", "post_date": "01/29/2015 01:22:34", "content": "I used make file under directory make under top dir:
\ncd make; make -f Cblas.cpu.mk
\nYou need add some stuff in it, not remember exactly what but here is the lines from my version
\nthat you can see what is missing:
\nexport LDFLAGS= -lcblas -lm -lz -lpthread `pkg-config --libs opencv`
\nCFLAGS += -DMSHADOW_USE_CBLAS=1 -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_CUDA=0
\nIt creates the bin in the same directory.
\nI assume you install Cuda developemnt kit by ignoring nvidia driver otherwise next time login your VM will be in a bad state. CPU takes close to 6hours to run all 45 iterations on i7-4710HQ and not uses more than 4GB memory and a single CPU.
\nMaybe there is way to make multthreading but I havent looked at it much.
\nIn terms of running on windows, my attempt for cxxnet was a failure since it uses heavily GNU specific stuff and combined with my weak Visual studio community experience so I couldn't manage to combine nvcc on Visual studio with gnu c++ on mingw nor VC++. On windows my only success was to be able run theano on python and GTX 860M runs a simple large matrix multiplication 24 times faster than on CPU. So I am looking to dual boot my laptop to use GPU on a linux flavor after many years of satisfied user of linux on VMs.
", "votes": null, "replies": [] }, { "id": 63339, "author_name": "telser", "author_url": "", "post_date": "01/30/2015 02:49:24", "content": "Quick question, Bing; is there an easy way to specify to hold out and report error rates on a validation set during training in the conf file?
\n\nThanks!
", "votes": null, "replies": [] }, { "id": 63357, "author_name": "mdjawad", "author_url": "", "post_date": "01/30/2015 13:31:21", "content": "[quote=kartal;61442]
\nHi Bing,
\nWhen I run the gen_train.py function I am getting these warnings "convert is not recognized as an internal or external command". Are you using a special python package to resize images?
\n[/quote]
\nInstall Imagemagick
", "votes": null, "replies": [] }, { "id": 63366, "author_name": "wolfganggross", "author_url": "", "post_date": "01/30/2015 14:56:59", "content": "Hi
\nI got CXXNET installed and it runs on the cpu. But on the gpu (Iuse dev = gpu) after initializing I get:
\ninitializing end, start working
update round 0
and it stops and jumps back to the shell promt. No error code, nothing. And no training.
\nI'm using Ubuntu 14.04 and the NVIDIA Geforce GTX 750 Ti.
\nCUDA seems to work and I even get:
\nUse CUDA Device 0: GeForce GTX 750 Ti
CXXNetTrainer, devCPU=0
after starting cxxnet.
\nAny idea what's the problem?
", "votes": null, "replies": [] }, { "id": 63368, "author_name": "telser", "author_url": "", "post_date": "01/30/2015 17:04:09", "content": "I'm working off a ubuntu 14.04 amazon AWS gpu instance (grid K520), and one of the things I had done prior to trying to work with cxxnet was go through http://tleyden.github.io/blog/2014/10/25/cuda-6-dot-5-on-aws-gpu-instance-running-ubuntu-14-dot-04/.
\nI hadn't tried to use cxxnet before doing the installation, though.
", "votes": null, "replies": [] }, { "id": 63370, "author_name": "akilaw", "author_url": "", "post_date": "01/30/2015 18:30:59", "content": "[quote=Wolf Gang;63366]
\nHi
\nI got CXXNET installed and it runs on the cpu. But on the gpu (Iuse dev = gpu) after initializing I get:
\ninitializing end, start working
update round 0
and it stops and jumps back to the shell promt. No error code, nothing. And no training.
\nI'm using Ubuntu 14.04 and the NVIDIA Geforce GTX 750 Ti.
\nCUDA seems to work and I even get:
\nUse CUDA Device 0: GeForce GTX 750 Ti
CXXNetTrainer, devCPU=0
after starting cxxnet.
\nAny idea what's the problem?
\n[/quote]
\nCheck this post by lametta. Doing that fixed the same issue for me https://www.kaggle.com/c/datasciencebowl/forums/t/11279/public-start-guide-of-deep-network-with-a-score-of-1-382285/62236#post62236
", "votes": null, "replies": [] }, { "id": 63426, "author_name": "mdjawad", "author_url": "", "post_date": "02/02/2015 03:43:40", "content": "[quote=Ertuka;63287]
\nI used make file under directory make under top dir:
\ncd make; make -f Cblas.cpu.mk
\nYou need add some stuff in it, not remember exactly what but here is the lines from my version
\nthat you can see what is missing:
\nexport LDFLAGS= -lcblas -lm -lz -lpthread `pkg-config --libs opencv`
\nCFLAGS += -DMSHADOW_USE_CBLAS=1 -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_CUDA=0
\nIt creates the bin in the same directory.
\nI assume you install Cuda developemnt kit by ignoring nvidia driver otherwise next time login your VM will be in a bad state. CPU takes close to 6hours to run all 45 iterations on i7-4710HQ and not uses more than 4GB memory and a single CPU.
\nMaybe there is way to make multthreading but I havent looked at it much.
\nIn terms of running on windows, my attempt for cxxnet was a failure since it uses heavily GNU specific stuff and combined with my weak Visual studio community experience so I couldn't manage to combine nvcc on Visual studio with gnu c++ on mingw nor VC++. On windows my only success was to be able run theano on python and GTX 860M runs a simple large matrix multiplication 24 times faster than on CPU. So I am looking to dual boot my laptop to use GPU on a linux flavor after many years of satisfied user of linux on VMs.
\n[/quote]
\nThank you very much, it helped!
", "votes": null, "replies": [] }, { "id": 63438, "author_name": "graydyn", "author_url": "", "post_date": "02/02/2015 15:19:43", "content": "[quote=Bing Xu;63204]
\nI use opencv to decode, opencv's output for jpg is 3 channel, and all are same.
\n[/quote]
\nHas anybody had any luck getting this to run with 1 channel? I've tried to reduce my channels to 1, and my image manipulation apps all say the image is greyscale, but no matter what CXXNet isn't working for me unless I set the channel dimension to 3.
\nI have this feeling like my image apps are lying to me, and that my JPGs really are still 3 channels. Not sure what to use to legitimately make a one channel image though.
", "votes": null, "replies": [] }, { "id": 63440, "author_name": "jfkingiii", "author_url": "", "post_date": "02/02/2015 15:46:07", "content": "They are 1 channel, per Bing's note cxxnet converts them to 3 channel no matter what.
\nGetting cxxnet to handle single channel input without replicating the data three times would be a good enhancement.
", "votes": null, "replies": [] }, { "id": 63491, "author_name": "yetiman", "author_url": "", "post_date": "02/03/2015 18:20:35", "content": "I haven't been following this thread, but thought it might be a good way to learn something about cxxnet to see how it compares with similar tools.
\nDidn't have much trouble building it (on Linux Mint 17), although I did have to monkey with the make file and the source code slightly (nothing that hasn't been covered in this forum already). Once it was built I managed to run the MNIST example/tutorial successfully (with and without GPU acceleration). Very nice! Compares very favorably to other CNN implementations as far as speed is concerned!
\nFirst I have a simple question: The MNIST config file has "pad = 1" for the first conv layer, but the cxxnet "Layers" documentation doesn't mention a "pad" option. What is it for?
\nAnd a comment/suggestion: It would be nice to have the ability to specify a hold-out or validation set in the config file (that either isn't part of the training data, or is a random N% of the training data) against which the model is tested after each training epoch, using the selected metric, rather than (or in addition to) evaluation vs. the training data. Would help with cross-validation and finding "early stopping" criteria.
", "votes": null, "replies": [] }, { "id": 63492, "author_name": "yetiman", "author_url": "", "post_date": "02/03/2015 19:04:55", "content": "Never mind my padding question above. I looked at the code and it appears to mean exactly the same thing as it does in Caffe. That was my assumption, but lacking documentation I wasn't sure.
", "votes": null, "replies": [] }, { "id": 63619, "author_name": "telser", "author_url": "", "post_date": "02/04/2015 22:33:46", "content": "If we're adding feature requests, I would note that it seems strange to have both a training config file and prediction config file,given the substantial overlap in parameters between the two.
", "votes": null, "replies": [] }, { "id": 63632, "author_name": "spintronic", "author_url": "", "post_date": "02/05/2015 07:46:13", "content": "", "votes": null, "replies": [] }, { "id": 63669, "author_name": "wanji28918", "author_url": "", "post_date": "02/06/2015 00:49:09", "content": "You can decode with `CV_LOAD_IMAGE_GRAYSCALE` or `CV_LOAD_IMAGE_UNCHANGED` parameter, that will give you a single channel image :)
\n[quote=Bing Xu;63204]
\nI use opencv to decode, opencv's output for jpg is 3 channel, and all are same.
\n[quote=Oleksandr Baiev;63180]
\nHi Bing,
\nThank you for sharing the guide.
\n
In the config file you set image shape to (3,48,48). As I can understand "3" is a number of input channels. But images in dataset is in gray-scale and have only one channel. How you convert from 1 to 3 channels? Is it just a copy?
[/quote]
\n[/quote]
", "votes": null, "replies": [] }, { "id": 63758, "author_name": "superfan123", "author_url": "", "post_date": "02/07/2015 22:16:35", "content": "After add path to header and lib files , I run "sh build.sh" and get a error message --
\n#error -- unsupported GNU version! gcc 4.9 and up are not supported!
\nIs there an easy way to get round this error?
\nThanks.
", "votes": null, "replies": [] }, { "id": 63760, "author_name": "songgc", "author_url": "", "post_date": "02/07/2015 22:58:24", "content": "[quote=superfan123;63758]
\nAfter add path to header and lib files , I run "sh build.sh" and get a error message --
\n#error -- unsupported GNU version! gcc 4.9 and up are not supported!
\nIs there an easy way to get round this error?
\nThanks.
\n[/quote]
\nThe root cause is that CUDA 6.5 doesn't support gcc 4.9 or up. gcc 4.8 should be good.
", "votes": null, "replies": [] }, { "id": 63761, "author_name": "esorar", "author_url": "", "post_date": "02/07/2015 23:13:11", "content": "Just modify CUDA config.h file to accept 4.9 since CUDA 6.5 contraints gcc to 4.8 and lower.
\nI have used gcc 4.9 on F21 with CUDA 6.5 and not seen any issue so far.
\n[quote=superfan123;63758]
\nAfter add path to header and lib files , I run "sh build.sh" and get a error message --
\n#error -- unsupported GNU version! gcc 4.9 and up are not supported!
\nIs there an easy way to get round this error?
\nThanks.
\n[/quote]
", "votes": null, "replies": [] }, { "id": 63762, "author_name": "superfan123", "author_url": "", "post_date": "02/07/2015 23:15:14", "content": "[quote=Guocong Song;63760]
\n[quote=superfan123;63758]
\nAfter add path to header and lib files , I run "sh build.sh" and get a error message --
\n#error -- unsupported GNU version! gcc 4.9 and up are not supported!
\nIs there an easy way to get round this error?
\nThanks.
\n[/quote]
\nThe root cause is that CUDA 6.5 doesn't support gcc 4.9 or up. gcc 4.8 should be good.
\n[/quote]
\nYour are right. I changed gcc link file in ~/CUDA-6.5/bin and make it point to gcc++-4.8 but that didn't fix the problem,
\nAnything else I should change?
\nI have gcc4.9 and 4.8 installed on Ubuntu 14.04.
\nThanks a lot.
", "votes": null, "replies": [] }, { "id": 63764, "author_name": "superfan123", "author_url": "", "post_date": "02/07/2015 23:26:59", "content": "[quote=Ertuka;63761]
\nJust modify CUDA config.h file to accept 4.9 since CUDA 6.5 contraints gcc to 4.8 and lower.
\nI have used gcc 4.9 on F21 with CUDA 6.5 and not seen any issue so far.
\n[quote=superfan123;63758]
\nAfter add path to header and lib files , I run "sh build.sh" and get a error message --
\n#error -- unsupported GNU version! gcc 4.9 and up are not supported!
\nIs there an easy way to get round this error?
\nThanks.
\n[/quote]
\n[/quote]
\nThanks Ertuka, Could you show me a sample statement to modify config.h?
", "votes": null, "replies": [] }, { "id": 63765, "author_name": "superfan123", "author_url": "", "post_date": "02/07/2015 23:34:03", "content": "Thanks guys, Now it's working :)
", "votes": null, "replies": [] }, { "id": 63766, "author_name": "superfan123", "author_url": "", "post_date": "02/08/2015 00:31:23", "content": "Now after add those header and lib --
\n-I/usr/local/cuda-6.5/include
\n-I/opt/intel/mkl/include
\n-I/opt/intel/composer_xe_2015.2.164/mkl/include
\n-L/opt/intel/mkl/lib/intel64
\n-L/opt/intel/composer_xe_2015.2.164/compiler/lib/intel64
\n-L/usr/local/cuda-6.5/lib64
\n\nI got lots of errors like --
\n/opt/intel/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `mkl_spblas_ccoo0nhuuc__mmout_par'
\n/opt/intel/mkl/lib/intel64/libmkl_intel_thread.so: undefined reference to `mkl_lapack_ps_dsymv_nb'
/opt/intel/mkl/lib/intel64/libmkl_intel_lp64.so: undefined reference to `mkl_lapack_chesvxx'
/opt/intel/mkl/lib/intel64/libmkl_intel_lp64.so: undefined reference to `mkl_lapack_ssygst'
I guess some header file is missing...
\nAppreciate your help!
", "votes": null, "replies": [] }, { "id": 63767, "author_name": "esorar", "author_url": "", "post_date": "02/08/2015 00:41:08", "content": "Looks like you don't have mkl, so build it as
\nbuild.sh blas=1
\nand install blas libs possibly atlas-devel.
", "votes": null, "replies": [] }, { "id": 63768, "author_name": "superfan123", "author_url": "", "post_date": "02/08/2015 00:52:59", "content": "I installed intel MKL composer package and I also include MKL header and lib. Maybe something else still missing.
\nThanks
", "votes": null, "replies": [] }, { "id": 63770, "author_name": "charlesroberson", "author_url": "", "post_date": "02/08/2015 04:46:04", "content": "It was going well. I built cxxnet with a small change to the Makefile. I tested on CPU with MNIST and it worked. It took some trial and error to get the original data location correct: I put in in cxxnet/example/kaggle_bowl/bowl/data and the new images (48x48) are in cxxnet/example/kaggle_bowl/data. (the difference is the extra bowl dir).
\nThe problem is in generating the image lists:
\npython gen_img_list.py train cxxnet/example/kaggle_bowl/bowl/sampleSubmission.csv data/train train.lst
\nit immediately gives the error no such file. When I touch a file of that name there to create an empty file, it stops due to nothing in the file.
\nDid I get the data locations correct?
\nHow do you create sampleSubmission.csv? Am I putting it in the correct place?
", "votes": null, "replies": [] }, { "id": 63774, "author_name": "nikeshbajaj", "author_url": "", "post_date": "02/08/2015 11:45:15", "content": "Does this tutorial giving score =1.3 for everybody, It is not working for me, I had score 4.7 by this tutorial. I am doing something wrong?
", "votes": null, "replies": [] }, { "id": 63775, "author_name": "superfan123", "author_url": "", "post_date": "02/08/2015 13:21:55", "content": "@Bing Xu,Thank you for this great tutorial!
\nI got 1.42 on LB after a sleepless night. Totally worth it.
\nI have a GTX610 with 48 cuda core and it takes 2 hour for modeling step. Plan to upgrade to a GPU and try it again. I need to sell one kidney to buy a K80. ;)
\nStill need to figure out MKL part. :-\\
", "votes": null, "replies": [] }, { "id": 63789, "author_name": "telser", "author_url": "", "post_date": "02/09/2015 01:40:28", "content": "[quote=Charles Roberson;63770]
\nIt was going well. I built cxxnet with a small change to the Makefile. I tested on CPU with MNIST and it worked. It took some trial and error to get the original data location correct: I put in in cxxnet/example/kaggle_bowl/bowl/data and the new images (48x48) are in cxxnet/example/kaggle_bowl/data. (the difference is the extra bowl dir).
\nThe problem is in generating the image lists:
\npython gen_img_list.py train cxxnet/example/kaggle_bowl/bowl/sampleSubmission.csv data/train train.lst
\nit immediately gives the error no such file. When I touch a file of that name there to create an empty file, it stops due to nothing in the file.
\nDid I get the data locations correct?
\nHow do you create sampleSubmission.csv? Am I putting it in the correct place?
\n[/quote]
\nsampleSubmission.csv is made available by Kaggle on the data page. You point the command at wherever you store that.
", "votes": null, "replies": [] }, { "id": 63905, "author_name": "hasselmann", "author_url": "", "post_date": "02/10/2015 09:49:59", "content": "Thanks Bing for the excellent script, I am trying to learn a bit about the CNNs that way.
\nI could get cxxnet running on cpu and without mkl and got through the example with the 48x48 resized pictures and got a score around 1.39 (after 10 hrs..). Then I tried instead of resizing with imagemagick (as in the example) to use a python script to generate 48x48 sized train files via cropping and finally stored the files using cv2.imwrite.
\nI have the same number of train images with the same 48x48 size as in the example, but generated in a different way. Now, the cxxnet file doesnt seem to learn anymore, here is a sample output what I get from cxxnet:
\n...
\nThreadImagePageIterator:image_list=./train.lst, bin=./train.bin
loading mean image from models/image_mean.bin
ThreadBufferIterator: buffer_size=2
initializing end, start working
round 0:[ 100] 467 sec elapsed[1] train-error:0.936077
round 1:[ 100] 1241 sec elapsed[2] train-error:0.936077
round 2:[ 100] 2014 sec elapsed[3] train-error:0.934785
round 3:[ 100] 2783 sec elapsed[4] train-error:0.936077
round 4:[ 100] 3566 sec elapsed[5] train-error:0.936077
round 5:[ 100] 4377 sec elapsed[6] train-error:0.934785
round 6:[ 100] 5122 sec elapsed[7] train-error:0.936077
...
Its similar to the problem description I found here:
\nhttps://github.com/antinucleon/cxxnet/issues/28
\nIs there something wrong in the way I generate the jpgs? Or should I adjust
some parameters in bowl.conf to achieve convergence? Right now I didnt touch bowl.conf,
its exactly as in the example.
@nils:
\nDid you regenerate the mean image file using the new set of training images that you created?
", "votes": null, "replies": [] }, { "id": 63978, "author_name": "charlesroberson", "author_url": "", "post_date": "02/10/2015 23:10:30", "content": "@Torgos, Thanks for the info. Of course that fixed it. I had more issues but fixed them myself and got it all working.
\n@Bing, Thanks for the work and sharing.
\nThis next suggestion can help EVERYBODY.
\n@All in CNN, The submission files are 4x to 7x bigger than the ones I used for my RF model submissions. Looking at the submission file, I see why. There are many entries with values below 0.0001 which look like 6.nnnnne-07.
\nIf all numbers less than 0.0001 (1% of 1%) were set to 0, that would significantly cut the data size: likely by 5x or better. This will translate into much smaller zip files as well. My current CNN submission zip files are 70 MB while my RF submission zip files are around 10MB to 15MB.
\nI'd do this myself, but I don't know Python and it looks like line 29 in the make_submission.py file needs changing.
\nI think doing this will help everybody and kaggle.com due to a reduction in bandwidth. Things are just going to heat up more in the next 34 days! Also, there are some other contests ending near the same day as this one.
", "votes": null, "replies": [] }, { "id": 64133, "author_name": "ajulian", "author_url": "", "post_date": "02/12/2015 18:15:57", "content": "Hi all,
\nI'm new to this forum (and not a data science expert either). First of all, I want to thank Bing Xu & Tianqi Chen for this nice cxxnet thing. It took me several unsuccessful days to compile opencv on my old 2009 Macbook (with Mavericks) until I realized I could use homebrew; the cxxnet path has been (quite) smooth since then; I am now where many of you were 40-50 days ago: score of 1.39, thinking on input augmentation...
\nMy problem is that my GPU has not enough memory (just 256MB) to run cxxnet with the given bowl.conf, and it took 10 hours to run 45 rounds in CPU mode.
\nIs there something I can try to run cxxnet in my GPU? I have reduced the resize parameters from 48*48 to 30*30, but still no luck. (BTW, I know GPU+CUDA works properly because the MNIST tutorial runs on it)
\nThanks!
", "votes": null, "replies": [] }, { "id": 64134, "author_name": "paweljankiewicz", "author_url": "", "post_date": "02/12/2015 18:21:48", "content": "Try reducing the batch size. Either way I recommend getting ec2 g2large instance. Spot requests are $0.10 per hour.
", "votes": null, "replies": [] }, { "id": 64135, "author_name": "ajulian", "author_url": "", "post_date": "02/12/2015 18:31:11", "content": "Thanks for your fast answer. I took a look to aws and google cloud but I did not understand how to proceed for this specific need: do you upload the code as-is to a virtual machine? or do you have to adapt the code to a virtual array of GPUs?
", "votes": null, "replies": [] }, { "id": 64136, "author_name": "paweljankiewicz", "author_url": "", "post_date": "02/12/2015 18:37:45", "content": "You gain access to your own instance. You can log and do everything with it through ssh.
\nWhat I did was
\n1. Create instance (ubuntu)
\n2. Attach ebs storage to it for persistant data
\n3. install everything : libraries, cuda. Save the image (ami) of the instance (it can be terminated any time so you need to make sure you can start it up again fast).
", "votes": null, "replies": [] }, { "id": 64156, "author_name": "akilaw", "author_url": "", "post_date": "02/13/2015 05:39:22", "content": "You can follow this blog or use the AMI they mentioned
\nhttp://tleyden.github.io/blog/2014/10/25/cuda-6-dot-5-on-aws-gpu-instance-running-ubuntu-14-dot-04/
\nUse spot instances. GPU spot instance cost is around 0.064
", "votes": null, "replies": [] }, { "id": 64195, "author_name": "hasselmann", "author_url": "", "post_date": "02/13/2015 21:59:15", "content": "[quote=zeros_benchmark;63973]
\n@nils:
\nDid you regenerate the mean image file using the new set of training images that you created?
\n[/quote]
\nI deleted all the files in the models folder so I assume the mean image was newly generated. I now also got cxxnet working on my image set-I simply did a clean compilation and generated all the cropped images again. I still dont know what went wrong before but it works now, thanks!
\nAnother thing, is it possible to view the filter banks within cxxnet? Is there some simple way to extract and visualize them from the models files?
", "votes": null, "replies": [] }, { "id": 65089, "author_name": "jiweiliu", "author_url": "", "post_date": "02/28/2015 01:58:09", "content": "When I try to use the code in cv, the prediction file sometimes lost some samples.
\nSay the test.lst have 10200 samples, which is correct, but test.txt only contains 10000 samples. How to deal with it?
\nSorry I didn't go through every page of this post. maybe my question is answered somewhere.
", "votes": null, "replies": [] }, { "id": 65090, "author_name": "telser", "author_url": "", "post_date": "02/28/2015 02:05:15", "content": "Change the batch size; cxxnet only predicts multiples of it.
\n\nhttps://github.com/antinucleon/cxxnet/issues/17
", "votes": null, "replies": [] }, { "id": 65091, "author_name": "jiweiliu", "author_url": "", "post_date": "02/28/2015 02:11:30", "content": "[quote=Torgos;65090]
\nChange the batch size; cxxnet only predicts multiples of it.
\nhttps://github.com/antinucleon/cxxnet/issues/17
\n[/quote]
\nI see. Thank you very much!
", "votes": null, "replies": [] }, { "id": 65092, "author_name": "binghsu", "author_url": "", "post_date": "02/28/2015 02:16:13", "content": "We will release v0.2 of CXXNET. It will fully fix this problem. And new version will support real time augmentation, multi-GPU training and distributed training. Stay tuned!
\n\n[quote=rcarson;65089]
\nWhen I try to use the code in cv, the prediction file sometimes lost some samples.
\nSay the test.lst have 10200 samples, which is correct, but test.txt only contains 10000 samples. How to deal with it?
\nSorry I didn't go through every page of this post. maybe my question is answered somewhere.
\n[/quote]
", "votes": null, "replies": [] }, { "id": 65098, "author_name": "maineiac", "author_url": "", "post_date": "02/28/2015 03:00:19", "content": "[quote=Bing Xu;65092]
\nWe will release v0.2 of CXXNET. It will fully fix this problem. And new version will support real time augmentation, multi-GPU training and distributed training. Stay tuned!
\n[/quote]
\n\nThat's awesome! I'm on the edge of my seat :)
\nWhat sort of real time augmentation are you going to include?
", "votes": null, "replies": [] }, { "id": 65112, "author_name": "shankarmsy", "author_url": "", "post_date": "02/28/2015 06:17:12", "content": "[quote=Bing Xu;65092]
\nWe will release v0.2 of CXXNET. It will fully fix this problem. And new version will support real time augmentation, multi-GPU training and distributed training. Stay tuned!
\n[quote=rcarson;65089]
\nWhen I try to use the code in cv, the prediction file sometimes lost some samples.
\nSay the test.lst have 10200 samples, which is correct, but test.txt only contains 10000 samples. How to deal with it?
\nSorry I didn't go through every page of this post. maybe my question is answered somewhere.
\n[/quote]
\n[/quote]
\nThat's terrific Bing. You got all the missing pieces covered with those updates.
", "votes": null, "replies": [] }, { "id": 65211, "author_name": "mconway", "author_url": "", "post_date": "03/02/2015 02:55:28", "content": "", "votes": null, "replies": [] }, { "id": 65212, "author_name": "mconway", "author_url": "", "post_date": "03/02/2015 02:56:37", "content": "[quote=Robert Griesmeyer;62324]
\nDid anyone get it running on the Macbook on OSX?
\n[/quote]
\nYes, use Blas and the following attached files for generating cxxnet and your images. Otherwise, you will get a mismatch in the shape of tensors. Total running time for training on a MacAir (CPU only) was ~5 hours with prediction time of about 15 minutes.
\nRegards,
\nMark
", "votes": null, "replies": [] }, { "id": 65401, "author_name": "dopelearner", "author_url": "", "post_date": "03/04/2015 11:47:27", "content": "get the following error when running ./build.sh.
cxxnet/io/cxxnet_data.cpp:6:28: error: mshadow/tensor.h: No such file or directory
In file included from cxxnet/io/cxxnet_data.cpp:18:
cxxnet/io/cxxnet_iter_cifar-inl.hpp:5:38: error: mshadow/tensor_container.h: No such file or directory
In file included from cxxnet/io/cxxnet_data.cpp:21:
cxxnet/io/cxxnet_iter_img-inl.hpp:14:30: error: opencv2/opencv.hpp: No such file or directory
This means that the header files are not being included correctly. I have modified the Makefile with -I option to include all the headers. Does anybody have any idea why this is happening?
\nI have already cloned mshadow and can see the header files in the directory but they are not being included for some reason.
\nPlease find attached my Makefile and error log.Thanks!
", "votes": null, "replies": [] }, { "id": 65402, "author_name": "dopelearner", "author_url": "", "post_date": "03/04/2015 11:50:43", "content": "^ Oops, The last Makefile(1.57 KB) is the correct one. Ignore the first one.
", "votes": null, "replies": [] }, { "id": 65559, "author_name": "", "author_url": "", "post_date": "03/06/2015 02:17:26", "content": "Hi Bing, thanx a lot for sharing! In the sample config file (bowl.conf), some conv layers don't have a stride value. So I'm wondering what the default stride is.
\nPlease advise. Many thanx in advance!
\n\nJJ
", "votes": null, "replies": [] }, { "id": 65627, "author_name": "telser", "author_url": "", "post_date": "03/06/2015 19:53:08", "content": "[quote=Bing Xu;65092]
\nWe will release v0.2 of CXXNET. It will fully fix this problem. And new version will support real time augmentation, multi-GPU training and distributed training. Stay tuned!
\n[/quote]
\nLooking forward to it for the diabetes competition; I don't suppose you're including a quick option in the config file to split (and report on) a % of data off for validation?
", "votes": null, "replies": [] }, { "id": 65631, "author_name": "binghsu", "author_url": "", "post_date": "03/06/2015 20:01:30", "content": "There is many ways to do it and we don't think it is an important feature to hard code.
\nSolution in V2:
\n1. sed -n '1, 20000p' train.lst > tr.lst
\nsed -n '20000, 30337p' train.lst > va.lst
\nThen generate bin according to it.
\n2. Use script in tools to split list file in to parts, like 6 separate bin files. Then set in configuration file which bins you want to use to train and which bins you want to treat as eval
\n\n3. Use python interface, do slice by numpy
\n[quote=Torgos;65627]
\n[quote=Bing Xu;65092]
\nWe will release v0.2 of CXXNET. It will fully fix this problem. And new version will support real time augmentation, multi-GPU training and distributed training. Stay tuned!
\n[/quote]
\nLooking forward to it for the diabetes competition; I don't suppose you're including a quick option in the config file to split (and report on) a % of data off for validation?
\n[/quote]
", "votes": null, "replies": [] }, { "id": 65677, "author_name": "dopelearner", "author_url": "", "post_date": "03/07/2015 11:54:39", "content": "I was able to get all the header files included but running ./build.sh is throwing errors like:
\ncxxnet/io/cxxnet_data.cpp:12: error: ‘real_t’ in namespace ‘mshadow’ does not name a type
\ncxxnet/io/cxxnet_iter_cifar-inl.hpp:14: error: ‘struct mshadow::Tensor<mshadow::cpu, 4, float>’ has no member named ‘dptr’
\ncxxnet/io/cxxnet_iter_cifar-inl.hpp:42: error: ‘struct mshadow::Tensor<mshadow::cpu, 4, float>’ has no member named ‘shape’
\nPlease find my Makefile attached. I have added all of the header file directories for OpenCV with the -I flag to CFLAGS because I had installed OpenCV in a custom location (no root access :/ ).
\n
The full error log is present in whatever.log
@Bing, can you take a quick look at this? Btw, thanks for writing CXXNET :)
", "votes": null, "replies": [] }, { "id": 65684, "author_name": "mconway", "author_url": "", "post_date": "03/07/2015 15:45:38", "content": "[quote=Saurav Gupta;65677]
\nI was able to get all the header files included but running ./build.sh is throwing errors like:
\ncxxnet/io/cxxnet_data.cpp:12: error: ‘real_t’ in namespace ‘mshadow’ does not name a type
\ncxxnet/io/cxxnet_iter_cifar-inl.hpp:14: error: ‘struct mshadow::Tensor<mshadow::cpu, 4, float>’ has no member named ‘dptr’
\ncxxnet/io/cxxnet_iter_cifar-inl.hpp:42: error: ‘struct mshadow::Tensor<mshadow::cpu, 4, float>’ has no member named ‘shape’
\nPlease find my Makefile attached. I have added all of the header file directories for OpenCV with the -I flag to CFLAGS because I had installed OpenCV in a custom location (no root access :/ ).
\n
The full error log is present in whatever.log
@Bing, can you take a quick look at this? Btw, thanks for writing CXXNET :)
\n[/quote]
\nYou may be using the newer version of the mshadow code (2.x). You have to point to the older version (1.x).
", "votes": null, "replies": [] }, { "id": 65771, "author_name": "alexeygolyshev", "author_url": "", "post_date": "03/09/2015 11:39:08", "content": "deleted
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "60426": "", "60429": "", "60431": "", "60439": "", "60440": "", "60441": "", "60464": "", "60483": "", "60499": "", "60503": "", "61418": "", "61419": "", "61420": "", "61421": "", "61423": "", "61426": "", "61427": "", "61428": "", "61432": "", "61433": "", "61436": "", "61437": "", "61438": "", "61442": "", "61443": "", "61446": "", "61447": "", "61471": "", "61497": "", "61503": "", "61522": "", "61523": "", "61528": "", "61575": "", "61581": "", "61597": "", "61598": "", "61599": "", "61600": "", "61601": "", "61632": "", "61633": "", "61726": "", "61732": "", "61733": "", "61783": "", "61786": "", "61792": "", "61793": "", "61828": "", "61830": "", "61848": "", "61849": "", "61887": "", "61917": "", "61919": "", "61929": "", "61934": "", "62072": "", "62073": "", "62083": "", "62150": "", "62194": "", "62204": "", "62214": "", "62236": "", "62285": "", "62286": "", "62302": "", "62324": "", "62328": "", "62332": "", "62334": "", "62346": "", "62347": "", "62378": "", "62410": "", "62509": "", "62815": "", "62917": "", "62936": "", "63180": "", "63204": "", "63208": "", "63220": "", "63272": "", "63287": "", "63339": "", "63357": "", "63366": "", "63368": "", "63370": "", "63426": "", "63438": "", "63440": "", "63491": "", "63492": "", "63619": "", "63632": "", "63669": "", "63758": "", "63760": "", "63761": "", "63762": "", "63764": "", "63765": "", "63766": "", "63767": "", "63768": "", "63770": "", "63774": "", "63775": "", "63789": "", "63905": "", "63973": "", "63978": "", "64133": "", "64134": "", "64135": "", "64136": "", "64156": "", "64195": "", "65089": "", "65090": "", "65091": "", "65092": "", "65098": "", "65112": "", "65211": "", "65212": "", "65401": "", "65402": "", "65559": "", "65627": "", "65631": "", "65677": "", "65684": "", "65771": "" }, "source": "meta" }