{"metadata":{"kaggle":{"accelerator":"none","dataSources":[{"sourceId":67836,"databundleVersionId":7527339,"sourceType":"competition"}],"dockerImageVersionId":30646,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.10.13"},"papermill":{"default_parameters":{},"duration":10609.387065,"end_time":"2024-02-09T18:57:34.988277","environment_variables":{},"exception":null,"input_path":"__notebook__.ipynb","output_path":"__notebook__.ipynb","parameters":{},"start_time":"2024-02-09T16:00:45.601212","version":"2.5.0"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"#### This notebook was inspired from [medium](https://medium.com/)###and [here](https://www.kaggle.com/code/ammarnassanalhajali/layout-parser-model-training) ##and [here](https://www.kaggle.com/code/salmankhondker/starter-notebook-dl-sprint-2-0)##and [here](https://www.kaggle.com/code/ataullhasaim/mws-dl-enigma-1-0-starter-notebook)","metadata":{"papermill":{"duration":0.004098,"end_time":"2024-02-09T16:00:48.361028","exception":false,"start_time":"2024-02-09T16:00:48.356930","status":"completed"},"tags":[]}},{"cell_type":"code","source":"#install detectron2\n!python -m pip install 'git+https://github.com/facebookresearch/detectron2.git'","metadata":{"execution":{"iopub.execute_input":"2024-02-09T16:00:48.370091Z","iopub.status.busy":"2024-02-09T16:00:48.369375Z","iopub.status.idle":"2024-02-09T16:02:52.802786Z","shell.execute_reply":"2024-02-09T16:02:52.801695Z"},"papermill":{"duration":124.440401,"end_time":"2024-02-09T16:02:52.805219","exception":false,"start_time":"2024-02-09T16:00:48.364818","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# detectron2 imports\nimport detectron2\nfrom detectron2.utils.logger import setup_logger\nsetup_logger()\nfrom detectron2 import model_zoo\nfrom detectron2.engine import DefaultTrainer\nfrom detectron2.engine import DefaultPredictor\nfrom detectron2.config import get_cfg\nfrom detectron2.utils.visualizer import Visualizer\nfrom detectron2.structures import BoxMode\nfrom detectron2.data import DatasetCatalog, MetadataCatalog\nfrom detectron2.evaluation import COCOEvaluator, inference_on_dataset, LVISEvaluator\nfrom detectron2.data import build_detection_test_loader\nfrom detectron2.utils.visualizer import ColorMode\n\n# other libs (other necessary imports in Colab file to make the list shorter here)\n\nimport torch, torchvision\nimport torchvision.transforms as transforms\nfrom pathlib import Path\nimport os\nfrom PIL import Image\nimport matplotlib.pyplot as plt\nfrom tqdm import tqdm\nimport pandas as pd\nimport cv2\nfrom pathlib import Path\nimport os","metadata":{"execution":{"iopub.execute_input":"2024-02-09T16:02:52.824727Z","iopub.status.busy":"2024-02-09T16:02:52.823919Z","iopub.status.idle":"2024-02-09T16:02:58.928071Z","shell.execute_reply":"2024-02-09T16:02:58.927028Z"},"papermill":{"duration":6.116522,"end_time":"2024-02-09T16:02:58.930604","exception":false,"start_time":"2024-02-09T16:02:52.814082","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#coco Data create and register\ndef create_data_pairs(input_path, detectron_img_path, detectron_annot_path, dir_type = 'train'):\n\n    img_paths = Path(input_path +'/images/train/').glob('*.jpg')\n\n    pairs = []\n    for img_path in img_paths:\n\n        file_name_tmp = str(img_path).split('/')[-1].split('.')\n        file_name_tmp.pop(-1)\n        file_name = '.'.join((file_name_tmp))\n\n        label_path = Path(input_path + '/labels/train/' + file_name + '.txt')\n\n        if label_path.is_file():\n\n            line_img = detectron_img_path + '/images/train/'+ file_name + '.jpg'\n            line_annot = detectron_annot_path+'/labels/train/' + file_name + '.txt'\n            pairs.append([line_img, line_annot])\n\n    return pairs\n\ninput_path = '/kaggle/input/dl-enigma-10-sust-cse-carnival-2024/dlenigma1/BadODD'\n\ndetectron_img_path = '/kaggle/input/dl-enigma-10-sust-cse-carnival-2024/dlenigma1/BadODD'\ndetectron_annot_path = '/kaggle/input/dl-enigma-10-sust-cse-carnival-2024/dlenigma1/BadODD'\n\ntrain = create_data_pairs(input_path, detectron_img_path, detectron_annot_path, 'train')\nval = create_data_pairs(input_path, detectron_img_path, detectron_annot_path, 'valid')","metadata":{"execution":{"iopub.execute_input":"2024-02-09T16:02:58.950991Z","iopub.status.busy":"2024-02-09T16:02:58.950014Z","iopub.status.idle":"2024-02-09T16:03:07.222825Z","shell.execute_reply":"2024-02-09T16:03:07.222005Z"},"papermill":{"duration":8.285235,"end_time":"2024-02-09T16:03:07.225190","exception":false,"start_time":"2024-02-09T16:02:58.939955","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def create_coco_format(data_pairs):\n    \n    data_list = []\n\n    for i, path in enumerate(data_pairs):\n        \n        filename = path[0]\n\n        img_h, img_w = cv2.imread(filename).shape[:2]\n\n        img_item = {}\n        img_item['file_name'] = filename\n        img_item['image_id'] = i\n        img_item['height']= img_h\n        img_item['width']= img_w\n\n        #print(str(i), filename)\n\n\n        annotations = []\n        with open(path[1]) as annot_file:\n            lines = annot_file.readlines()\n            for line in lines:\n                if line[-1]==\"\\n\":\n                  box = line[:-1].split(' ')\n                else:\n                  box = line.split(' ')\n\n                class_id = box[0]\n                x_c = float(box[1])\n                y_c = float(box[2])\n                width = float(box[3])\n                height = float(box[4])\n\n                x1 = (x_c - (width/2)) * img_w\n                y1 = (y_c - (height/2)) * img_h\n                x2 = (x_c + (width/2)) * img_w\n                y2 = (y_c + (height/2)) * img_h\n\n                annotation = {\n                    \"bbox\": list(map(float,[x1, y1, x2, y2])),\n                    \"bbox_mode\": BoxMode.XYXY_ABS,\n                    \"category_id\": int(class_id),\n                    \"iscrowd\": 0\n                }\n                annotations.append(annotation)\n            img_item[\"annotations\"] = annotations\n        data_list.append(img_item)\n    return data_list \n\ntrain_list = create_coco_format(train)\nval_list = create_coco_format(val)","metadata":{"execution":{"iopub.execute_input":"2024-02-09T16:03:07.244740Z","iopub.status.busy":"2024-02-09T16:03:07.244021Z","iopub.status.idle":"2024-02-09T16:09:26.505605Z","shell.execute_reply":"2024-02-09T16:09:26.504771Z"},"papermill":{"duration":379.273842,"end_time":"2024-02-09T16:09:26.507994","exception":false,"start_time":"2024-02-09T16:03:07.234152","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for catalog_name, file_annots in [(\"train\", train_list), (\"val\", val_list)]:\n    DatasetCatalog.register(catalog_name, lambda file_annots = file_annots: file_annots)\n    MetadataCatalog.get(catalog_name).set(thing_classes=['auto_rickshaw', 'bicycle', 'bus', 'car', 'cart_vehicle', 'construction_vehicle', 'motorbike', 'person', 'priority_vehicle', 'three_wheeler', 'train', 'truck', 'wheelchair'])\n\nmetadata = MetadataCatalog.get(\"train\") ","metadata":{"execution":{"iopub.execute_input":"2024-02-09T16:09:26.528568Z","iopub.status.busy":"2024-02-09T16:09:26.527995Z","iopub.status.idle":"2024-02-09T16:09:26.533654Z","shell.execute_reply":"2024-02-09T16:09:26.532820Z"},"papermill":{"duration":0.016938,"end_time":"2024-02-09T16:09:26.535527","exception":false,"start_time":"2024-02-09T16:09:26.518589","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Custom augmentation function\ndef custom_mapper(dataset_dict):\n     dataset_dict = copy.deepcopy(dataset_dict)\n     image = utils.read_image(dataset_dict[\"file_name\"], format=\"BGR\")\n\n     transform_list = [T.RandomBrightness(0.5, 1.2),\n                      T.RandomFlip(prob=0.5, horizontal=False, vertical=True),\n                      T.RandomFlip(prob=0.5, horizontal=True, vertical=False)\n                      ]\n     image, transforms = T.apply_transform_gens(transform_list, image)\n\n     dataset_dict[\"image\"] = torch.as_tensor(\n        image.transpose(2, 0, 1).astype(\"float32\"))\n\n     annos = [\n        utils.transform_instance_annotations(obj, transforms, image.shape[:2])\n        for obj in dataset_dict.pop(\"annotations\")\n        if obj.get(\"iscrowd\", 0) == 0\n     ]\n     instances = utils.annotations_to_instances(annos, image.shape[:2])\n\n     dataset_dict[\"instances\"] = utils.filter_empty_instances(instances)\n     return dataset_dict","metadata":{"execution":{"iopub.execute_input":"2024-02-09T16:09:26.553958Z","iopub.status.busy":"2024-02-09T16:09:26.553222Z","iopub.status.idle":"2024-02-09T16:09:26.561163Z","shell.execute_reply":"2024-02-09T16:09:26.560289Z"},"papermill":{"duration":0.018945,"end_time":"2024-02-09T16:09:26.562921","exception":false,"start_time":"2024-02-09T16:09:26.543976","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#training\ncfg = get_cfg()\ncfg.merge_from_file(model_zoo.get_config_file(\"COCO-Detection/faster_rcnn_R_50_FPN_3x.yaml\"))\ncfg.DATASETS.TRAIN = (\"train\",)\ncfg.DATALOADER.NUM_WORKERS = 2\ncfg.MODEL.DEVICE = 'cuda' # cuda\ncfg.MODEL.WEIGHTS = \"detectron2://COCO-Detection/faster_rcnn_R_50_FPN_3x/137849458/model_final_280758.pkl\"\ncfg.SOLVER.IMS_PER_BATCH = 2\ncfg.SOLVER.CHECKPOINT_PERIOD = 750\ncfg.SOLVER.WARMUP_ITERS = 500\ncfg.SOLVER.GAMMA = 0.05\ncfg.SOLVER.BASE_LR = 0.0005\ncfg.DATALOADER.AUGMENTATIONS = [(\"CustomMapper\", custom_mapper),]\ncfg.SOLVER.MAX_ITER = 350 # (train_size / batch_size) * 100\ncfg.MODEL.ROI_HEADS.BATCH_SIZE_PER_IMAGE = 256 # 512\ncfg.MODEL.ROI_HEADS.NUM_CLASSES = len(MetadataCatalog.get(\"train\").thing_classes)\ncfg.SOLVER.STEPS = (20500, )\n\nos.makedirs(cfg.OUTPUT_DIR, exist_ok=True)\ntrainer = DefaultTrainer(cfg)\ntrainer.resume_or_load(resume=False)\n\nimport time as t\ns1 = t.time()\ntry:\n  trainer.train()\nexcept:\n  None\ns2 = t.time()\nprint(s2-s1)","metadata":{"execution":{"iopub.execute_input":"2024-02-09T16:09:26.580850Z","iopub.status.busy":"2024-02-09T16:09:26.580585Z","iopub.status.idle":"2024-02-09T18:53:02.713574Z","shell.execute_reply":"2024-02-09T18:53:02.712706Z"},"papermill":{"duration":9816.144174,"end_time":"2024-02-09T18:53:02.715503","exception":false,"start_time":"2024-02-09T16:09:26.571329","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cfg = get_cfg()\ncfg.merge_from_file(model_zoo.get_config_file(\"COCO-Detection/faster_rcnn_R_50_FPN_3x.yaml\"))\ncfg.MODEL.DEVICE = 'cuda' # cuda\ncfg.MODEL.ROI_HEADS.NUM_CLASSES = 13\ncfg.MODEL.WEIGHTS = \"/kaggle/working/output/model_final.pth\"\ncfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5   # set the testing threshold for this model\npredictor = DefaultPredictor(cfg)\n\nim = cv2.imread(\"/kaggle/input/dl-enigma-10-sust-cse-carnival-2024/dlenigma1/BadODD/images/test/chittagong_bohoddarhat1_1003.jpg\")\noutputs = predictor(im)\nv = Visualizer(im, scale=1., instance_mode =  ColorMode.IMAGE    )\n\nv = v.draw_instance_predictions(outputs[\"instances\"].to(\"cpu\"))\nimg = v.get_image()[:,:,[2,1,0]]\nimg = Image.fromarray(img)\nplt.figure(figsize=(10, 10))\nplt.imshow(img)","metadata":{"execution":{"iopub.execute_input":"2024-02-09T18:53:03.017741Z","iopub.status.busy":"2024-02-09T18:53:03.016439Z","iopub.status.idle":"2024-02-09T18:53:05.268503Z","shell.execute_reply":"2024-02-09T18:53:05.267553Z"},"papermill":{"duration":2.409975,"end_time":"2024-02-09T18:53:05.276009","exception":false,"start_time":"2024-02-09T18:53:02.866034","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_prediction_string(boxes, scores, classes,height1, width1):\n    pred_strs = []\n    for i, score in enumerate(scores):\n        single_pred_str = \"\"\n        single_pred_str += str(float(classes[i])) + \" \" + str(float(score)) + \" \"\n        #print(boxes[i])\n        x_center , y_center, width,height = boxes[i]\n        x1 = ((float(x_center) + float(width))/2)/float(width1)\n        y1 = ((float(y_center) + float(height))/2)/float(height1)\n        \n        width = (-float(x_center) + float(width))/float(width1)\n        height = (-float(y_center) + float(height))/float(height1)\n        x2=float(x1)-(float(width)/2)\n        y2=float(y1)-(float(height)/2)\n        #single_pred_str += \" \".join(str(float(x)) for x in boxes[i])\n        single_pred_str +=  f\"{x2} {y2} {width} {height}\"\n        \n        pred_strs.append(single_pred_str)\n    ans = ','.join(map(str, pred_strs))\n    if len(ans):\n        return ans\n#     the solution metrics faield in case of a NaN, '' (empty). So, return \"0 0 0 0 0 0\" for NaN, '' (empty) string\n    return \"0 0 0 0 0 0\"\n\n# get the predcition in id, ImageID, PredictionString_pred foramt ...\n\ndef get_prediction_entry(i, filename, boxes, scores, classes,height1, width1):\n    return {\n        \"id\": i, # strating from 0 ...\n        \"ImageID\": filename.split('.')[0], # before the extension ...\n        \"PredictionString_pred\": get_prediction_string(boxes, scores, classes,height1, width1)\n    }\n\n# Directory path ...\ntest_directory = \"/kaggle/input/dl-enigma-10-sust-cse-carnival-2024/dlenigma1/BadODD/images/test\"\n#test_directory = \"/kaggle/input/image-test\"\n# Load the model ...\n#model = YOLO('/kaggle/working/runs/detect/train/weights/best.pt')\n\n# do the inference ...\n\ndef predict_all_files(test_directory):\n    predictions = []\n    for i,filename in tqdm(enumerate(os.listdir(test_directory))):\n        if filename.endswith(\".jpg\"):\n            filepath = os.path.join(test_directory, filename)\n            im = cv2.imread(filepath)\n            height1, width1, _ = im.shape\n            \n            #print(im.shape)\n            results = predictor(im)\n            instances = results[\"instances\"]\n            boxes_cpu = instances.pred_boxes.tensor.cpu()\n            scores_cpu = instances.scores.cpu()\n            classes_cpu = instances.pred_classes.cpu()\n\n                   # Convert CPU tensors to NumPy arrays\n            boxes = boxes_cpu.numpy()  # Extract bounding boxes as a numpy array\n            scores = scores_cpu.numpy()  # Extract prediction scores as a numpy array\n            classes = classes_cpu.numpy()\n            prediction = get_prediction_entry(i, filename, boxes, scores, classes,height1, width1)\n            predictions.append(prediction)\n            #print(predictions[0])\n            \n#             to csv format ...\n    predictions_df = pd.DataFrame(predictions)\n    predictions_df.to_csv(\"submission.csv\", index=False)\n\n# call the inference function ...\npredict_all_files(test_directory)","metadata":{"execution":{"iopub.execute_input":"2024-02-09T18:53:05.588244Z","iopub.status.busy":"2024-02-09T18:53:05.587294Z","iopub.status.idle":"2024-02-09T18:57:31.933791Z","shell.execute_reply":"2024-02-09T18:57:31.932794Z"},"papermill":{"duration":266.50347,"end_time":"2024-02-09T18:57:31.935894","exception":false,"start_time":"2024-02-09T18:53:05.432424","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]}]}