{"cells":[{"metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","trusted":true},"cell_type":"code","source":"import os\nimport numpy as np \nimport pandas as pd \nimport cv2\nimport matplotlib.pyplot as plt\nimport re\nfrom collections import defaultdict","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#Constants\nCONFIDENCE_THRESHOLD = 0.1 # Filter predicted bboxes\n\n#Seeds\nSEED = 42\nnp.random.seed(seed=SEED)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Global Wheat Detection. Predict with pre-trained YOLO v4 and Darknet.\n\nThis notebook is dedicated to making predictions for Global Wheat Detection competition. I saw some YOLO implementation based on PyTorch or TensorFlow, but I will show you how to do it only with Darknet, without additional frameworks. ","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Make darknet","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"**DEPRECATED**\n\n**This approach will not allow you to use the GPU in Kaggle Kernles, but it works fine for CPU**\n\nUsually, you can just ```! git clone``` files from repositories, but this competition prohibits the internet access of submission kernels. If you want to use the internet, check the 4-th version of this notebook.\nWe will use additional files from ```global_wheat_detection_models```:\n* ```darknet``` - copy of [AlexeyAB's Darknet repository](https://github.com/AlexeyAB/darknet)\n* ```competition_files``` - some files from [my repository for this competition](https://github.com/Gooogr/Kaggle_Global_Wheat_Detection). We need custom cfg. file and script for result log.txt file. But you can also find there training Google Colab notebook.\n* ```yolov4.weights``` - pre-trained weights on COCO dataset from AlexeyAB's repository. We will use it to check our darknet build.\n* ```yolov4_naive.weights``` - current version of my pre-trained weights from Google Colab.\n\n","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"**RUNNING WITH GPU** \n\nThis method was discovered by Mark Perg, this is the link to his noteboook: https://www.kaggle.com/markpeng/darknet-gpu-on-kaggle<br>\nWe will use a pre-built darknet binaries with additional files (libdarknet.so and minimal example files for ```dog.jpg```  predict)","execution_count":null},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"## Use version from github\n# ! git clone https://github.com/AlexeyAB/darknet.git \n\n# Use Darknet with CPU and make it from source\n# ! cp -a /kaggle/input/global-wheat-detection-models/darknet/darknet/. /kaggle/darknet/\n\n## Use pre-built Daknet binaries with GPU support\n! cp -a /kaggle/input/global-wheat-detection-models/darknet_gpu_prebuilt/darknet_gpu_prebuilt/. /kaggle/darknet/","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# %cd /kaggle/darknet\n\n# ## Uncomment if you want to use Darknet with GPU.\n\n# !sed -i 's/OPENCV=0/OPENCV=1/' Makefile\n# # !sed -i 's/GPU=0/GPU=1/' Makefile\n# # !sed -i 's/CUDNN=0/CUDNN=1/' Makefile\n# # !sed -i 's/OPENMP=0/OPENMP=1/' Makefile\n\n# !head Makefile\n\n# %%capture \n# #Use  %%caputure to hide huge terminal output\n# ! make clean\n# ! make --silent","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Test our darknet build.<br>\nWe can everything for that by default, except weights. Copy them  from additional files (you can also find them in darknet folder). It will be pre-trained weights from AlexeyAB's github trained on COCO dataset.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"! mkdir /kaggle/darknet/weights\n! cp -a /kaggle/input/global-wheat-detection-models/yolov4.weights /kaggle/darknet/weights","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%capture \n%cd /kaggle/darknet\n! chmod 777 ./darknet\n! ./darknet detect cfg/yolov4.cfg weights/yolov4.weights data/dog.jpg -dont_show","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_preds = cv2.imread('predictions.jpg')\nfig, ax = plt.subplots(figsize=(7, 7))\nax.imshow(sample_preds)\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Good, now let's move to our custom model predictions.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Setting up custom config files and weights\n","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Create ```my_files``` folder inside darknet directory for nessesary files. Copy prepaired files and add pre-trained weights.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"! ls /kaggle/input/global-wheat-detection-models/competition_files/competition_files","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"yolov4.cfg file, txt2json script and weights","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"! mkdir /kaggle/darknet/my_files\n# cfg file and txt2json\n! cp -a /kaggle/input/global-wheat-detection-models/competition_files/competition_files/. /kaggle/darknet/my_files\n# yolo weights (CHANGE LINK TO YOUR WEIGHTS HERE IF YOU NEED)\n! cp -a /kaggle/input/global-wheat-detection-models/yolov4_naive.weights /kaggle/darknet/weights","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"!mv /kaggle/darknet/my_files/yolov4-custom.cfg /kaggle/darknet/my_files/yolov4.cfg ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%cd /kaggle/darknet/my_files","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"obj.names","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"%%writefile obj.names\nWheat head","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"yolo.data","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"%%writefile yolo.data\n#classses = 1\nnames = /kaggle/darknet/my_files/obj.names","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"predict.txt","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def create_path_file(files_dir, save_dir):\n    %cd /kaggle/working/ \n    # from https://stackoverflow.com/questions/9816816/get-absolute-paths-of-all-files-in-a-directory\n    file = open(os.path.join(save_dir, \"predict.txt\"), \"w\")\n    for root, dirs, files in os.walk(os.path.abspath(files_dir)):\n        for item in files:\n            row = os.path.join(root, item)\n            file.write(row)\n            file.write('\\n')\n    file.close()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"create_path_file(files_dir='/kaggle/input/global-wheat-detection/test', \n                 save_dir='/kaggle/darknet/my_files/')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"! head /kaggle/darknet/my_files/predict.txt ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Predict","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Let's test our config on a separate image from test set","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"! ls /kaggle/darknet","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%capture \n%cd /kaggle/darknet\n\n! ./darknet detector test \\\nmy_files/yolo.data \\\nmy_files/yolov4.cfg \\\nweights/yolov4_naive.weights \\\n/kaggle/input/global-wheat-detection/test/2fd875eaa.jpg -dont_show","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_preds = cv2.imread('predictions.jpg')\nfig, ax = plt.subplots(figsize=(7, 7))\nax.imshow(sample_preds)\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Generate log file with boundary boxes for full test dataset","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"%%capture \n%cd /kaggle/darknet\n\n! ./darknet detector test \\\nmy_files/yolo.data \\\nmy_files/yolov4.cfg \\\nweights/yolov4_naive.weights \\\n-dont_show -ext_output < my_files/predict.txt > log.txt","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# # Uncomment to see result log file\n# ! cat log.txt","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Make submission","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Now we have log.txt with all bboxes in it. Time to parse this data to panads data frame.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def txt2json(file_path):\n    file_lines = open(file_path, 'r').read()\n    table_dict = defaultdict()\n    current_jpg_name = ''\n\n    jpg_delimiters = \" \", \"/\", \":\"\n    jpg_regexPattern = '|'.join(map(re.escape, jpg_delimiters))\n\n    for line in file_lines.splitlines():\n        if '.jpg' in line:\n            for item in re.split(jpg_regexPattern, line):\n                if '.jpg' in item:\n                    current_jpg_name = item\n                    table_dict[item] = []\n        if '%' in line:\n            split_string = (re.findall('-?\\d+', line))\n            split_string = list(filter(lambda x: x != \"\", split_string)) # remove empty strings from list\n            int_string = list(map(int, split_string))\n            sub_dict_keys = ['proba_%', 'left_x', 'top_y', 'width', 'height']\n            table_dict[current_jpg_name].append(dict(zip(sub_dict_keys, int_string)))\n    return table_dict","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"data = txt2json('/kaggle/darknet/log.txt')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# data['empty_sample'] = list()  #ONLY FOR NEGATIVE TEST, DON'T UNCOMMENT","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"img_id, proba, left_x, top_y, width, height = list([]), [], [], [], [], []\nfor key in data.keys():\n    try:\n        df = pd.DataFrame(data[key])\n        img_id.extend([key] * len(df))\n        proba.extend(df['proba_%'].values)\n        left_x.extend(df['left_x'].values)\n        top_y.extend(df['top_y'].values)\n        width.extend(df['width'].values)\n        height.extend(df['height'].values)\n    except: # in case of no detections\n        img_id.extend([key])\n        proba.extend([np.nan])\n        left_x.extend([np.nan])\n        top_y.extend([np.nan])\n        width.extend([np.nan])\n        height.extend([np.nan])\n\nresult_df = pd.DataFrame(list(zip(img_id, proba, left_x, top_y, width, height)), \n                         columns = ['img', 'proba_%', 'left_x', 'top_y', 'width', 'height'])\nresult_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Convert our submission to the final form","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_submission = pd.read_csv('/kaggle/input/global-wheat-detection/sample_submission.csv')\nsample_submission.head().T","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"result_df['proba_ratio'] = result_df['proba_%'] / 100","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def format_list(confidence, x, y, width, height):\n    temp_list =  [confidence, x, y, width, height]\n    if not np.isnan(confidence):\n        return ' '.join(str(item) for item in temp_list)\n    else:\n        return np.nan","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"result_df['sub_list'] = result_df.apply(lambda x: format_list(x.proba_ratio, \n                                                              x.left_x, \n                                                              x.top_y, \n                                                              x.width, \n                                                              x.height), axis = 1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"filter_condition = (result_df['proba_ratio'] > CONFIDENCE_THRESHOLD) | (result_df['proba_ratio'].isna())\nresult_df = result_df[filter_condition]\n\nresult_df.fillna('', inplace=True)\n\nimg_pred_list = []\nfor img_name in result_df['img'].unique():\n    img_pred_list.append(' '.join(str(item) for item in result_df[result_df['img']==img_name].sub_list))\n\nimg_names = [item.split('.')[0] for item in result_df['img'].unique()]\n\nsubmission = pd.DataFrame(zip(img_names, img_pred_list), \n                          columns = ['image_id', 'PredictionString'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission.to_csv('/kaggle/working/submission.csv', index=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### References\n\n* [Yolov4-darknet-Inference](https://www.kaggle.com/pabloberhauser/yolov4-darknet-inference)\n* [darknet-gpu-on-kaggle](https://www.kaggle.com/markpeng/darknet-gpu-on-kaggle)\n* [Darknet](https://github.com/AlexeyAB/darknet/)","execution_count":null},{"metadata":{"trusted":false},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}