# 8th Place Solution 

First of all I want to thank everyone who participated, shared kernels and was part of activate discussion.
PROBLEM:
As you have already heard and read many top team solutions have used very simple 2 STEP approach in fact this was heavily discussed here 
(
[https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/100815](https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/100815)
) and I also partially mentinoed in my kernel (
[https://www.kaggle.com/drhabib/starter-kernel-for-0-79](https://www.kaggle.com/drhabib/starter-kernel-for-0-79)
) :
STEP 1:  pretrain on 2015 data 
STEP 2:  finetune on 2019 data
This approach is good. But When I personally started experimenting I was extremely worried that our local CV was around 0.931 but LB was 0.811. This was a huge gap and we were afraid about shake up. Also just pertaining on old data using 2019 as Validation results in LB 0.75. Which indicated that our model was not generalizing good.
SOLUTION:
Our idea was very simple instead of training in two diffrent stages we will just combine all the data from 2019 and 2015 and try to work from here to improve our generalization.
PREPROCESSING.
ISSUE 1:
Remove black background.
For Image preprocessing and have used script and info from this awesome kernel
[https://www.kaggle.com/ratthachat/aptos-updatedv14-preprocessing-ben-s-cropping](https://www.kaggle.com/ratthachat/aptos-updatedv14-preprocessing-ben-s-cropping)
Thank you
[@ratthachat](/ratthachat)
The script basically removes all the black background.
```
def resize_to(img, targ_sz:int, use_min:bool=False):
    h,w = img.shape[:2]
    min_sz = (min if use_min else max)(w,h)
    ratio = targ_sz/min_sz
    return int(w*ratio),int(h*ratio)
def crop_image_from_gray(img,tol=7):
    if img.ndim ==2:
        mask = img>tol
        return img[np.ix_(mask.any(1),mask.any(0))]
    elif img.ndim==3:
        gray_img = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
        mask = gray_img>tol
check_shape = img[:,:,0][np.ix_(mask.any(1),mask.any(0))].shape[0]
    if (check_shape == 0): # image is too dark so that we crop out everything,
        return img # return original image
    else:
        img1=img[:,:,0][np.ix_(mask.any(1),mask.any(0))]
        img2=img[:,:,1][np.ix_(mask.any(1),mask.any(0))]
        img3=img[:,:,2][np.ix_(mask.any(1),mask.any(0))]
        img = np.stack([img1,img2,img3],axis=-1)

    return img
```
```
    def load_ben_color(path, size):
         image = cv2.imread(str(path))
          image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
          image = crop_image_from_gray(image)
          target_size = resize_to(image, size, use_min=True)
           image = cv2.resize(image, target_size)
           return PIL.Image.fromarray(image)
``
The only think I modified I added
resize_to
function to preserve original aspect ratio so the images look natural.  (My function called
load_ben_color` it has nothing to do with ben processing i was just too lazy to change name)
ISSUE 2:
Removing confusing labels and duplications.
Since we were combining 2015 and 2019 in to one dataset. It was really important to remove all the duplicates and confusing labels to prevent any leak. We have used this awesome kernel to do the job (
[https://www.kaggle.com/h4211819/more-information-about-duplicate](https://www.kaggle.com/h4211819/more-information-about-duplicate)
) Thank you
[@h4211819](/h4211819)
ISSUE 3.
Solving zoom problem. 
After all images were preprocessed to remove black background we observe, also heavily discussed on the forums that 2019 data looked zoomed to the center (below is the example).
![](assets/52a9d720d71b56ad33944e98.png)
Since we did not know how private data was we decided this problem is best solved using Augmentation random zooms to center, in this way we were making sure that we are not overfitting and also generalizing better.
AUGMENTATION:
This was very important part our solution (I posted experiments that I have performed with augmentations here:
[https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/100815](https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/100815)
)
Below is our augmentation:
Rotation
to 360,
flips
,
zoom
up to 1.35x ,
lightning
. This is how its looked:
![](assets/e3865fefacd5ca978417aa73.png)
TRAINING:
All the training was done using one cycle learning policy (fastai) with gradual increasing of size , we start with low resolution images and gradually go to higher resolution with reusing weights. You can find details on the my Github notebooks
[here](https://github.com/DrHB/APTOS-2019-GOLD-MEDAL-SOLUTION)
(I will keep updating) .
Model 1
:
Results:
Model:          EfficientNet-B5
Loss :          MSE
CV Sratagy   :  5 FOLD StratifiedKFold
After combining both datasets 2015 + 2019  and applying above mentioned tricks we immediately observed that our Local CV matches very well public scores.
```
EXP 740:
5 FOLD CV: 0.823
       LB: 0.821
       PB: 0.926
```
Model 2
:
Results:
```
Model:          EfficientNet-B4
Loss :          MSE
CV Sratagy   :  5 FOLD StratifiedKFold
```
data same as Model 1
```
EXP 765:
5 FOLD CV: 0.831
       LB: 0.816
PB: 0.927
```
This looked all promising our Local CV was stable and had good correlation with LB score . But stil we wanted to push further. And last think I decided to try is do pseudo labeling:
I took average prediction for 2019 test data from
EXP_740
and
EXP_765
. Added pseudo label data to train data and retrain
EXP_740
and
EXP_765
for 3 epoch below are results
```
EXP_740_PSD:
5 FOLD CV: 0.831
       LB: 0.93
       PB: 0.930
EXP_765_PSD:
5 FOLD CV: 0.83
       LB: 0.835
       PB: 0.929
```
The combined average of this two models results
0.931
on PB and
0.830
on LB =)
CONCLUSION:
This was very fun competition, as you can see we did not use many tricks just very systematical logical thinking and trusted our local CV =)