![\"\"](\"http://yunopon.sakura.ne.jp/sblo_files/wordroid/image/ttt00107.png\")
|\n|circle-2|![\"\"](\"http://yunopon.sakura.ne.jp/sblo_files/wordroid/image/ttt00108.png\")
|\n|circle-3|![\"\"](\"http://yunopon.sakura.ne.jp/sblo_files/wordroid/image/ttt00109.png\")
|Cifar-10 RBF 2 features ensemble [LB 0.9304]
\n\nIn Cifar-10 we tried the ensemble method with Digit Recognizer. Please see also 'Does RBF kernel with two fixed Landmarks work well in ensemble?'.
\n\n(Submitting the output submit.csv of this kernel with 'CIFAR-10 - Object Recognition in Images' gives 0.9304.)
\n\nThe input of network is 32x32x3, since the image data of Cifar-10 is an ordinary RGB image.
\n\n|...|MODEL|INPUT|OUTPUT|\n|-------------|:-------------|-----:|-----:|\n|1| image converter | 32 x 32 x 3 | 512 |\n|2| classifier | 512 | 10 |
\n\nIn this trial with Cifar-10, we try to make the configuration of the classifier different from Digit Recognizer.
\n\n|...|LAYER|INPUT|OUTPUT|DESC|\n|--:|:-------------|---:|---:|:-----|\n|LAYER-1| GaussianKernel(RBF) | 512 | 2 | .|\n|LAYER-2| GaussianKernel(RBF) | 2 | 10 | 10 fixed Landmarks |
\n\nDigit Recognizer used fixed values for the two landmarks of LAYER-1, but here we use a fixed value for the 10 landmarks of LAYER-2. The landmark of LAYER-1 will be trained.
\n\nThe 10 landmarks used are shown below. We change this landmarks little by little during 300 epochs. While maintaining the positional relationship of individual landmarks, we gradually reduce the radius of the circle. When using circle-3 from the beginning it did not converge well.
\n\n|LM|fixed 10 landmarks|\n|--:|:--:|\n|circle-1||\n|circle-2||\n|circle-3||
Let's see the output of LAYER-1 and the landmark of LAYER-2. You can see that the output of LAYER-1 is distributed around the landmark of LAYER-2.
\n\n|state|image|\n|--:|:--:|\n|10001|![\"\"](\"http://yunopon.sakura.ne.jp/sblo_files/wordroid/image/ttt00110.png\")
|
Individual learner only use two variables at classifier. Since we are trying to represent feature values with two variables, the bias is high and it looks like a weak learner. Therefore, the generalization performance of each learner is not so high.
\n\nThe results of ensemble of these individual results are very good.
\n\n|state|train data
f1_score|test data
kaggle LB|\n|:--|--:|--:|\n|10001|0.96747|0.87210|\n|10002|0.96433|0.87170|\n|10003|0.95208|0.86900|\n||||\n|3 learners above|0.99123|0.91760|\n||||\n|10004|0.95641|0.87220|\n|10005|0.95259|0.86310|\n|10006|0.95889|0.87910|\n|6 learners above|0.99444|0.92530|\n|9 learners|0.99534|0.93040|
Cifar-10 RBF 2 features ensemble [LB 0.9304]
\n\nIn Cifar-10 we tried the ensemble method with Digit Recognizer. Please see also 'Does RBF kernel with two fixed Landmarks work well in ensemble?'.
\n\n(Submitting the output submit.csv of this kernel with 'CIFAR-10 - Object Recognition in Images' gives 0.9304.)
\n\nThe input of network is 32x32x3, since the image data of Cifar-10 is an ordinary RGB image.
\n\n|...|MODEL|INPUT|OUTPUT|\n|-------------|:-------------|-----:|-----:|\n|1| image converter | 32 x 32 x 3 | 512 |\n|2| classifier | 512 | 10 |
\n\nIn this trial with Cifar-10, we try to make the configuration of the classifier different from Digit Recognizer.
\n\n|...|LAYER|INPUT|OUTPUT|DESC|\n|--:|:-------------|---:|---:|:-----|\n|LAYER-1| GaussianKernel(RBF) | 512 | 2 | .|\n|LAYER-2| GaussianKernel(RBF) | 2 | 10 | 10 fixed Landmarks |
\n\nDigit Recognizer used fixed values for the two landmarks of LAYER-1, but here we use a fixed value for the 10 landmarks of LAYER-2. The landmark of LAYER-1 will be trained.
\n\nThe 10 landmarks used are shown below. We change this landmarks little by little during 300 epochs. While maintaining the positional relationship of individual landmarks, we gradually reduce the radius of the circle. When using circle-3 from the beginning it did not converge well.
\n\n|LM|fixed 10 landmarks|\n|--:|:--:|\n|circle-1||\n|circle-2||\n|circle-3||
Let's see the output of LAYER-1 and the landmark of LAYER-2. You can see that the output of LAYER-1 is distributed around the landmark of LAYER-2.
\n\n|state|image|\n|--:|:--:|\n|10001||
Individual learner only use two variables at classifier. Since we are trying to represent feature values with two variables, the bias is high and it looks like a weak learner. Therefore, the generalization performance of each learner is not so high.
\n\nThe results of ensemble of these individual results are very good.
\n\n|state|train data
f1_score|test data
kaggle LB|\n|:--|--:|--:|\n|10001|0.96747|0.87210|\n|10002|0.96433|0.87170|\n|10003|0.95208|0.86900|\n||||\n|3 learners above|0.99123|0.91760|\n||||\n|10004|0.95641|0.87220|\n|10005|0.95259|0.86310|\n|10006|0.95889|0.87910|\n|6 learners above|0.99444|0.92530|\n|9 learners|0.99534|0.93040|