{"metadata":{"kaggle":{"accelerator":"none","dataSources":[{"sourceId":67836,"databundleVersionId":7527339,"sourceType":"competition"}],"dockerImageVersionId":30646,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.10.13"},"papermill":{"default_parameters":{},"duration":null,"end_time":null,"environment_variables":{},"exception":null,"input_path":"__notebook__.ipynb","output_path":"__notebook__.ipynb","parameters":{},"start_time":"2024-02-03T21:23:40.049899","version":"2.5.0"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"#### This notebook was inspired from [medium](https://medium.com/)###and [here](https://www.kaggle.com/code/ammarnassanalhajali/layout-parser-model-training) ##and [here](https://www.kaggle.com/code/salmankhondker/starter-notebook-dl-sprint-2-0)##and [here](https://www.kaggle.com/code/ataullhasaim/mws-dl-enigma-1-0-starter-notebook)","metadata":{}},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"code","source":"!python -m pip install 'git+https://github.com/facebookresearch/detectron2.git'","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# detectron2 imports\nimport detectron2\nfrom detectron2.utils.logger import setup_logger\nsetup_logger()\nfrom detectron2 import model_zoo\nfrom detectron2.engine import DefaultTrainer\nfrom detectron2.engine import DefaultPredictor\nfrom detectron2.config import get_cfg\nfrom detectron2.utils.visualizer import Visualizer\nfrom detectron2.structures import BoxMode\nfrom detectron2.data import DatasetCatalog, MetadataCatalog\nfrom detectron2.evaluation import COCOEvaluator, inference_on_dataset, LVISEvaluator\nfrom detectron2.data import build_detection_test_loader\nfrom detectron2.utils.visualizer import ColorMode\n\n# other libs (other necessary imports in Colab file to make the list shorter here)\n\nimport torch, torchvision\nimport torchvision.transforms as transforms","metadata":{"papermill":{"duration":5.453398,"end_time":"2024-02-03T21:25:08.240348","exception":false,"start_time":"2024-02-03T21:25:02.786950","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-02-04T12:15:34.265038Z","iopub.execute_input":"2024-02-04T12:15:34.265465Z","iopub.status.idle":"2024-02-04T12:15:40.775735Z","shell.execute_reply.started":"2024-02-04T12:15:34.265376Z","shell.execute_reply":"2024-02-04T12:15:40.774658Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from pathlib import Path","metadata":{"papermill":{"duration":0.016806,"end_time":"2024-02-03T21:25:08.265659","exception":false,"start_time":"2024-02-03T21:25:08.248853","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-02-04T12:15:40.777224Z","iopub.execute_input":"2024-02-04T12:15:40.777728Z","iopub.status.idle":"2024-02-04T12:15:40.784625Z","shell.execute_reply.started":"2024-02-04T12:15:40.777698Z","shell.execute_reply":"2024-02-04T12:15:40.783263Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#coco Data create and register\ndef create_data_pairs(input_path, detectron_img_path, detectron_annot_path, dir_type = 'train'):\n\n img_paths = Path(input_path +'/images/train/').glob('*.jpg')\n\n pairs = []\n for img_path in img_paths:\n\n file_name_tmp = str(img_path).split('/')[-1].split('.')\n file_name_tmp.pop(-1)\n file_name = '.'.join((file_name_tmp))\n\n label_path = Path(input_path + '/labels/train/' + file_name + '.txt')\n\n if label_path.is_file():\n\n line_img = detectron_img_path + '/images/train/'+ file_name + '.jpg'\n line_annot = detectron_annot_path+'/labels/train/' + file_name + '.txt'\n pairs.append([line_img, line_annot])\n\n return pairs\n\ninput_path = '/kaggle/input/dl-enigma-10-sust-cse-carnival-2024/dlenigma1/BadODD'\n\ndetectron_img_path = '/kaggle/input/dl-enigma-10-sust-cse-carnival-2024/dlenigma1/BadODD'\ndetectron_annot_path = '/kaggle/input/dl-enigma-10-sust-cse-carnival-2024/dlenigma1/BadODD'\n\ntrain = create_data_pairs(input_path, detectron_img_path, detectron_annot_path, 'train')\nval = create_data_pairs(input_path, detectron_img_path, detectron_annot_path, 'valid')","metadata":{"papermill":{"duration":12.495185,"end_time":"2024-02-03T21:25:20.769264","exception":false,"start_time":"2024-02-03T21:25:08.274079","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-02-04T12:15:40.786762Z","iopub.execute_input":"2024-02-04T12:15:40.787075Z","iopub.status.idle":"2024-02-04T12:15:59.060534Z","shell.execute_reply.started":"2024-02-04T12:15:40.787048Z","shell.execute_reply":"2024-02-04T12:15:59.059419Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import cv2","metadata":{"papermill":{"duration":0.015743,"end_time":"2024-02-03T21:25:20.793429","exception":false,"start_time":"2024-02-03T21:25:20.777686","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-02-04T12:15:59.061828Z","iopub.execute_input":"2024-02-04T12:15:59.062202Z","iopub.status.idle":"2024-02-04T12:15:59.067878Z","shell.execute_reply.started":"2024-02-04T12:15:59.062171Z","shell.execute_reply":"2024-02-04T12:15:59.066157Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def create_coco_format(data_pairs):\n \n data_list = []\n\n for i, path in enumerate(data_pairs):\n \n filename = path[0]\n\n img_h, img_w = cv2.imread(filename).shape[:2]\n\n img_item = {}\n img_item['file_name'] = filename\n img_item['image_id'] = i\n img_item['height']= img_h\n img_item['width']= img_w\n\n #print(str(i), filename)\n\n\n annotations = []\n with open(path[1]) as annot_file:\n lines = annot_file.readlines()\n for line in lines:\n if line[-1]==\"\\n\":\n box = line[:-1].split(' ')\n else:\n box = line.split(' ')\n\n class_id = box[0]\n x_c = float(box[1])\n y_c = float(box[2])\n width = float(box[3])\n height = float(box[4])\n\n x1 = (x_c - (width/2)) * img_w\n y1 = (y_c - (height/2)) * img_h\n x2 = (x_c + (width/2)) * img_w\n y2 = (y_c + (height/2)) * img_h\n\n annotation = {\n \"bbox\": list(map(float,[x1, y1, x2, y2])),\n \"bbox_mode\": BoxMode.XYXY_ABS,\n \"category_id\": int(class_id),\n \"iscrowd\": 0\n }\n annotations.append(annotation)\n img_item[\"annotations\"] = annotations\n data_list.append(img_item)\n return data_list \n\ntrain_list = create_coco_format(train)\nval_list = create_coco_format(val)","metadata":{"papermill":{"duration":448.723695,"end_time":"2024-02-03T21:32:49.525554","exception":false,"start_time":"2024-02-03T21:25:20.801859","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-02-04T12:15:59.070056Z","iopub.execute_input":"2024-02-04T12:15:59.070941Z","iopub.status.idle":"2024-02-04T12:24:24.201444Z","shell.execute_reply.started":"2024-02-04T12:15:59.070896Z","shell.execute_reply":"2024-02-04T12:24:24.199598Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for catalog_name, file_annots in [(\"train\", train_list), (\"val\", val_list)]:\n DatasetCatalog.register(catalog_name, lambda file_annots = file_annots: file_annots)\n MetadataCatalog.get(catalog_name).set(thing_classes=['auto_rickshaw', 'bicycle', 'bus', 'car', 'cart_vehicle', 'construction_vehicle', 'motorbike', 'person', 'priority_vehicle', 'three_wheeler', 'train', 'truck', 'wheelchair'])\n\nmetadata = MetadataCatalog.get(\"train\") ","metadata":{"papermill":{"duration":0.184084,"end_time":"2024-02-03T21:32:49.866611","exception":false,"start_time":"2024-02-03T21:32:49.682527","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-02-04T12:24:24.203645Z","iopub.execute_input":"2024-02-04T12:24:24.204157Z","iopub.status.idle":"2024-02-04T12:24:24.211899Z","shell.execute_reply.started":"2024-02-04T12:24:24.204095Z","shell.execute_reply":"2024-02-04T12:24:24.210386Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os","metadata":{"papermill":{"duration":0.159995,"end_time":"2024-02-03T21:32:50.178993","exception":false,"start_time":"2024-02-03T21:32:50.018998","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-02-04T12:24:24.213767Z","iopub.execute_input":"2024-02-04T12:24:24.214227Z","iopub.status.idle":"2024-02-04T12:24:24.227560Z","shell.execute_reply.started":"2024-02-04T12:24:24.214194Z","shell.execute_reply":"2024-02-04T12:24:24.226195Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#training\ncfg = get_cfg()\ncfg.merge_from_file(model_zoo.get_config_file(\"COCO-Detection/faster_rcnn_R_50_FPN_3x.yaml\"))\ncfg.DATASETS.TRAIN = (\"train\",)\ncfg.DATALOADER.NUM_WORKERS = 2\ncfg.MODEL.DEVICE = 'cuda' # cuda\ncfg.MODEL.WEIGHTS = \"detectron2://COCO-Detection/faster_rcnn_R_50_FPN_3x/137849458/model_final_280758.pkl\"\ncfg.SOLVER.IMS_PER_BATCH = 2\ncfg.SOLVER.CHECKPOINT_PERIOD = 500\ncfg.SOLVER.WARMUP_ITERS = 500\ncfg.SOLVER.GAMMA = 0.0005\ncfg.SOLVER.BASE_LR = 0.005\ncfg.SOLVER.MAX_ITER = 2 # (train_size / batch_size) * 100\ncfg.MODEL.ROI_HEADS.BATCH_SIZE_PER_IMAGE = 256 # 512\ncfg.MODEL.ROI_HEADS.NUM_CLASSES = len(MetadataCatalog.get(\"train\").thing_classes)\ncfg.SOLVER.STEPS = (20500, )\n\nos.makedirs(cfg.OUTPUT_DIR, exist_ok=True)\ntrainer = DefaultTrainer(cfg)\ntrainer.resume_or_load(resume=False)\n\nimport time as t\ns1 = t.time()\ntry:\n trainer.train()\nexcept:\n None\ns2 = t.time()\nprint(s2-s1)","metadata":{"papermill":{"duration":null,"end_time":null,"exception":false,"start_time":"2024-02-03T21:32:50.331392","status":"running"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

for inferencing follow this notebook

\n

detectron2 inference

","metadata":{}}]}