{"cells":[{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"import gc #garbage collector\nimport ast # operate with string representation of list\nimport os\n\nimport pandas as pd\nimport numpy as np\nimport cv2\nfrom scipy.stats import shapiro # for normal distribution checks \n\nimport plotly.express as px\nimport plotly.graph_objects as go\nimport plotly\nplotly.offline.init_notebook_mode(connected = True)\n\nimport matplotlib.pyplot as plt","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Global Wheat Detection. EDA\n\nThis notebook is dedicated to exploratory data analysis for [Global Wheat Detection](https://www.kaggle.com/c/global-wheat-detection) competition. The main focus will be on visualizing the differences between datasets from different sources and selecting key components for further data augmentation.\n","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Utilities","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def convert_coords(bbox):\n    '''\n    Transform boundary box coordinates from pandas dataframe to cv2.rectangle values\n    Pandas df values: x, y  width, height\n    '''\n    x, y, width, height = bbox\n    start_point = x, y\n    end_point = (x + width), (y + height)\n    return start_point, end_point\n\ndef plot_samples(df, img_ids=None, threshold=6, title=''):\n    '''\n    Plot image grid from seleted dataframe \n    https://stackoverflow.com/questions/46615554/how-to-display-multiple-images-in-one-figure-correctly/46616645\n    '''\n    if img_ids is None:\n        img_ids = df['image_id_ext'].unique()[:threshold]\n    cols = 3\n    rows = len(img_ids) // cols + 1\n    fig = plt.figure(figsize = (15, 5 * rows))\n    for i, img_id in enumerate(img_ids):\n        bboxes_list = df[df['image_id_ext'] == img_id].bbox.to_list()\n        img = cv2.imread(os.path.join(TRAIN_DIR_PATH, img_id))\n        for item in bboxes_list:\n            bbox = list(map(int, ast.literal_eval(item)))\n            strart_point, end_point = convert_coords(bbox)\n            color = (255, 0, 0) #RGB\n            thickness = 2\n            img = cv2.rectangle(img, strart_point, end_point, color, thickness)\n        fig.add_subplot(rows, cols, i+1)\n        plt.imshow(img)\n    plt.suptitle(title, fontsize=16)\n    fig.tight_layout()\n    fig.subplots_adjust(top=0.95)\n    plt.show()\n    \n# Took ideas of these functions from https://www.kaggle.com/aleksandradeis/globalwheatdetection-eda\n\ndef get_image_brightness(image_id_ext):\n    img = cv2.imread(os.path.join(TRAIN_DIR_PATH, image_id_ext))\n    # convert to grayscale\n    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)\n    # get average brightness\n    return np.array(gray).mean()\n\ndef get_percentage_of_green_pixels(image_id_ext):\n    img = cv2.imread(os.path.join(TRAIN_DIR_PATH, image_id_ext))\n    # convert to HSV\n    hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)\n    \n    # get the green mask\n    hsv_lower = (40, 40, 40) \n    hsv_higher = (70, 255, 255)\n    green_mask = cv2.inRange(hsv, hsv_lower, hsv_higher)\n    \n    return float(np.sum(green_mask)) / 255 / (1024 * 1024)\n\ndef get_percentage_of_yellow_pixels(image_id_ext):\n    img = cv2.imread(os.path.join(TRAIN_DIR_PATH, image_id_ext))\n    # convert to HSV\n    hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)\n    \n    # get the green mask\n    hsv_lower = (25, 40, 40) \n    hsv_higher = (35, 255, 255)\n    yellow_mask = cv2.inRange(hsv, hsv_lower, hsv_higher)\n    \n    return float(np.sum(yellow_mask)) / 255 / (1024 * 1024)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Constants","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"MAIN_PATH = '/kaggle/input/global-wheat-detection/'\nTRAIN_DIR_PATH = '/kaggle/input/global-wheat-detection/train/'\nTEST_DIR_PATH = '/kaggle/input/global-wheat-detection/test/'","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Boundary boxes analysis","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Train images amount', len(os.listdir(os.path.join(MAIN_PATH, 'train'))))\nprint('Test images amount', len(os.listdir(os.path.join(MAIN_PATH, 'test'))))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Note, that test dataset extremely small. Train dataset quite small as well, so I will have to use data augmentation.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df = pd.read_csv(os.path.join(MAIN_PATH, 'train.csv'))\nsample_submission = pd.read_csv(os.path.join(MAIN_PATH, 'sample_submission.csv'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df.head().T","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_submission.head().T","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Add additional column to train dataset with ```.jpg``` extention","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df['image_id_ext'] = train_df['image_id'] + '.jpg'","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's see on the diversity of boundary boxes amounts.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"ser = train_df['image_id']\n\nfig = px.histogram(ser, title = 'Amunt of bundary boxes on the each picture', \n                   labels={'x':'image id', 'y':'bbox amount'})\nfig.update_xaxes(categoryorder='total descending')\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"temp_df = train_df.copy()\ntemp_df['dummy_column'] = 1\nser = temp_df.groupby(['image_id']).sum()['dummy_column']","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = px.histogram(ser, title = 'Sum distribution of bboxes amount', \n                   labels={'y':'bbox amount'})\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Pictures usually have ~35 bboxes. Also, we have something looks like a nice normal distribution.<br>\nWe can use this values to validate our results: distribution on valdation dataset should be the same.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Shapiro-Wilk test for normality result\\n statistic:{:.3f}, p-value:{:.3E}'.format(*shapiro(ser.to_list())))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Yep, normal distribution.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"del temp_df\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's check how many pictures from train set don't have any bboxes at all","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"print('How many no-bbox images does exist in train dataset?', \n      len(os.listdir(TRAIN_DIR_PATH)) - len(train_df['image_id'].unique()))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Save their ids into additional set","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"no_bbox_img_ids = set(os.listdir(TRAIN_DIR_PATH)) - set((train_df['image_id_ext']).unique().tolist())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Checking the size distribution of boundary boxes. ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\n# convert column '[x, y, width, height]' to the separate pandas dataframe\nsplitted_data = train_df['bbox'].str.split(r'[^\\d.]+')\nbbox_df = pd.DataFrame.from_dict(dict(zip(splitted_data.index,splitted_data))).T\nbbox_df.drop(columns = [0, 5], inplace = True) #drop empty columns\nbbox_df.columns = ['x', 'y', 'bwidth', 'bheight']\nbbox_df = bbox_df.astype(float)\nbbox_df['size'] = bbox_df['bwidth'] * bbox_df['bheight']","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df = train_df.join(bbox_df)\ntrain_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df['size'].describe()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We definately have some anomalies here","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"max_bbox_size = train_df.groupby(['image_id']).max()['size'].to_frame()\n\nfig = px.histogram(max_bbox_size, title = 'Max distribution of bboxes area size', \n                   labels={'y':'bbox area size'})\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"MAX_ANOMALY_THRESHOLD = 120000\nprint('Size of anomaly frame: ', train_df[train_df['size'] > MAX_ANOMALY_THRESHOLD].shape)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_samples(train_df[train_df['size'] > MAX_ANOMALY_THRESHOLD], \n             threshold=15, \n             title='Images with anomaly big bboxes')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"min_bbox_size = train_df.groupby(['image_id']).min()['size'].to_frame()\n\nfig = px.histogram(min_bbox_size, title = 'Min distribution of bboxes area size', \n                   labels={'y':'bbox area size'})\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"MIN_ANOMALY_THRESHOLD = 1000\nprint('Size of anomaly frame: ', train_df[train_df['size'] < MIN_ANOMALY_THRESHOLD].shape)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_samples(train_df[train_df['size'] < MIN_ANOMALY_THRESHOLD], \n             threshold=9, \n             title = 'Images with anomaly small bboxes')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Thus, only bboxes of excessive size are dangerous anomalies in the dataset. There are few of them and they can be removed without damaging the generality of the model.<br>\nUltra-small boundary boxes can be explained either by very careful tracing of plants and by zooming the photos during the creation of the dataset. Anyway it will not hurt the model quality. <br>\nThis is a signal for us, that we should use this crop tric during data augmentation as well.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"sum_bbox_areas =  train_df.groupby(['image_id']).sum()['size']/(1024 * 1024)\n\nfig = px.histogram(sum_bbox_areas, title = 'Percentage distribution of image box coverage', \n                   labels={'y':'bbox area size'})\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"And again, we got something similar to the normal distribution, like it was in distribution of bboxes amount.\nWe can expect similar distribution on validation and test (if we will find additional data) datasets.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Shapiro-Wilk test for normality result \\n statistic:{:.3f}, p-value:{:.3E}'.format(*shapiro(sum_bbox_areas.to_list())))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Source distribution","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"According to the description of the competition and the transcription of acronyms from the dataset, the sources of the image are: \n* ```arvalis``` - ARVALIS, Institut du vegetal, France. \n* ```ethz``` - ETHZ, Swiss Federal Institute of Technology in Zurich, Switzerland\n* ```rres``` - Rothamsted Research Institute, UK\n* ```usask``` - University of Saskatchewan, Canada\n* ```inrae``` - INRAE, National Research Institute for Agriculture, Food and Environment, France","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = px.histogram(train_df.groupby(['image_id', 'source']).mean().index.to_frame()['source'],\n                   title = 'Image sources distribution',\n                   labels={'x':'source name', 'y':'images amount'})\nfig.update_xaxes(categoryorder='total descending')\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We will review the image samples from the each source below.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Dataset samples","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"**Samples with the maximum amount of boundary boxes.**","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"max_bbox_ids = train_df['image_id_ext'].value_counts().index.to_list()[:6]\nplot_samples(train_df, img_ids=max_bbox_ids, title='Samples with the maximum amount of boundary boxes')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Boundary boxes looks correct, but I will separetely check their areas later. Some boxes and wheat heads overlap, that can cause false positive errors. ","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"**Samples with the minimum amount of boundary boxes.**","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"min_bbox_ids = train_df['image_id_ext'].value_counts(ascending=True).index.to_list()\nplot_samples(train_df, img_ids=min_bbox_ids[:6], title='Samples with the minimum amount of boundary boxes')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Looks like the second and the third photos are serial. <br>\nYou can also see that the unmarked photos are pictures of the ground that has been trampled. Let's dive a little deeper into this problem, maybe some significant parts of dataset have this  problem?","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Sources of top 100 photost with the least amount of bboxes: ', \n      train_df[train_df['image_id_ext'].isin(min_bbox_ids[:100])]['source'].unique())    ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = px.histogram(train_df[train_df['source'] == 'arvalis_3'].groupby(['image_id']).mean()['size'], \n                   title = 'Percentage distribution of image box coverage in arvalis_3 dataset')                 \nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Yes, we have  problem images, but overall dataset behaviour the same with summary dataset.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"**Samples from different sources**","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"sources_list = list(train_df['source'].unique())\nprint('Complete list of sources: \\n', sources_list)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_samples(train_df[train_df['source'] == 'arvalis_1'], \n             title='Source: ARVALIS (Institut du vegetal is an applied agricultural research)')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_samples(train_df[train_df['source'] == 'arvalis_2'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_samples(train_df[train_df['source'] == 'arvalis_3'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_samples(train_df[train_df['source'] == 'inrae_1'],\n             title='Source: INRAE (National Research Institute for Agriculture, Food and Environment)')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_samples(train_df[train_df['source'] == 'ethz_1'],\n             title='Source: ETHZ (Swiss Federal Institute of Technology in Zurich)')\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_samples(train_df[train_df['source'] == 'rres_1'],\n             title='Soure: Rothamsted Research Institute')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_samples(train_df[train_df['source'] == 'usask_1'],\n             title='Source: University of Saskatchewan')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"It can be seen that different plant species are typical for different sources. Moreover, the photos were taken at different times of the day, from different angles and at different stages of harvest ripening.<br>\nPerhaps you should add a separate classifier to the model to determine the type of plants and train the appropriate model for it.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"**Empty images without boundary boxes**","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_samples(train_df, img_ids=list(no_bbox_img_ids)[:6], \n             title='Examples of empty images without boundary boxes')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"All of this  image (all 49, I mean) are very dark and can be haracterize by empty ground. ","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Color hystograms","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"In view of the large difference in the quality of images of the dataset, consider the distribution of their brightness, perhaps there will be anomalies or patterns.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\n#Calculate mean brightness\nser = train_df.groupby(['image_id_ext']).mean().reset_index()['image_id_ext']\nmean_brightness = ser.apply(get_image_brightness)\n# Add results to train_df\nbright_df = pd.DataFrame({'image_id_ext': ser, 'mean brightness': mean_brightness})\ntrain_df = train_df.merge(bright_df, on='image_id_ext')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Brightness (min - max)\nfig = px.histogram(mean_brightness, title = 'Mean brightness distribution')\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Well, we can see the bimodal distribution. There are 2 large groups of images with similar brightness in the dataset.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"first_group_id = train_df[(train_df['mean brightness'] >= 76) & (train_df['mean brightness'] <= 78)]['image_id_ext'].unique()\nsecond_group_id = train_df[(train_df['mean brightness'] >= 106) & (train_df['mean brightness'] <= 108)]['image_id_ext'].unique()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_samples(train_df, img_ids=first_group_id[:6], \n             title='Examples of images with brightness [76 - 78]')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_samples(train_df, img_ids=second_group_id[:6], \n             title='Examples of images with brightness [106 - 108]')\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sorted_bright_images = train_df.groupby(['image_id_ext']).mean()['mean brightness']\\\n                       .sort_values(ascending=False).index.to_list()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_samples(train_df, img_ids=sorted_bright_images[:6], \n             title='The most bright images')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_samples(train_df, img_ids=sorted_bright_images[:-7:-1], \n             title='The most dark images')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Different brightness corresponds to different datasets and different plant species.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Idea to look at color distributions was taken  from [here](https://www.kaggle.com/aleksandradeis/globalwheatdetection-eda).<br>\nThe point was that plant with different  maturnity levels has different green pixels saturated. In this case we can find a kink between color and size of wheat heads. In the other hand empty pictures would have gray dominant pixels. ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\n#Calculate color percentage \nser = train_df.groupby(['image_id_ext']).mean().reset_index()['image_id_ext']\ngreen_percentage = ser.apply(get_percentage_of_green_pixels)\nyellow_percentage = ser.apply(get_percentage_of_yellow_pixels)\n# Add results to train_df\ncolors_df = pd.DataFrame({'image_id_ext': ser, 'green %': green_percentage, 'yellow %': yellow_percentage})\ntrain_df = train_df.merge(colors_df, on='image_id_ext')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"gc.collect()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"yellow_means = train_df.groupby(['image_id']).mean()['yellow %']\ngreen_means = train_df.groupby(['image_id']).mean()['green %']","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = go.Figure()\nfig.add_trace(go.Histogram(x=yellow_means, marker_color='#eeff00'))\nfig.add_trace(go.Histogram(x=green_means, marker_color='#55ff00'))\n\n# Reduce opacity to see both histograms\nfig.update_traces(opacity=0.7)\nfig.update_layout(\n    #barmode='overlay',\n    title_text='Yellow and green percentage distribution', \n    xaxis_title_text='% of colored pixels', # xaxis label\n    yaxis_title_text='Count', # yaxis label\n    #bargap=0.2, # gap between bars of adjacent location coordinates\n    #bargroupgap=0.1 # gap between bars of the same location coordinates\n)\n\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sorted_yellow_images = train_df.groupby(['image_id_ext']).mean()['yellow %']\\\n                       .sort_values(ascending=False).index.to_list()\nsorted_green_images = train_df.groupby(['image_id_ext']).mean()['green %']\\\n                       .sort_values(ascending=False).index.to_list()\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_samples(train_df, img_ids=sorted_yellow_images[:3], \n             title='The most yellow images')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_samples(train_df, img_ids=sorted_yellow_images[:-4:-1], \n             title='The least yellow images')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_samples(train_df, img_ids=sorted_green_images[:3], \n             title='The most green images')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_samples(train_df, img_ids=sorted_green_images[:-4:-1], \n             title='The least green images')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"markdown","source":"# Conclusions\n\n*Dataset*\n\n* The images of plants in the dataset vary greatly in brightness and number of boxes. What's more, biological species are just as different.\n* A large number of targets and boundary boxes overlapping each other. It increases the risk of false negative error.\n* The distribution of the number of targets and the area of boxes relative to the image are subject to normal distribution. This can be used for model quality control during validation.\n* Training dataset are relatively small for this competition, data augmentation will be critical part.\n\n\n*Data Augmentation*\n\nWhat might work:\n* Flipping images horizontally and vertically\n* Crop-resize\n* Gamma, contrast and brightness tuning","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Acknowledgments\nNotebooks, that was useful  for this research:\n* [GlobalWheatDetection EDA](https://www.kaggle.com/aleksandradeis/globalwheatdetection-eda)\n* [GWD: EDA + Starter Code](https://www.kaggle.com/yashchoudhary/gwd-eda-and-starter-code-beginner-friendly)","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}