{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# CACHED: Context-Aware Chart Element Detection\n\n\n- Authors: Pengyu Yan, Saleem Ahmed, David Doermann\n- Paper: https://arxiv.org/abs/2305.04151\n- GitHub: https://github.com/pengyu965/ChartDete\n\nCACHED is an object detection model for charts. It gives bounding boxes for chat elements, title, tick labels, etc. Since I had hard times running this model, I share the compiled packages. This notebook is just about the systems (compile, install and run), and not about machine learning. I didn't train for this competition and all the data are unchanged from authors model.","metadata":{}},{"cell_type":"markdown","source":"\n\n## LICENSE\n\n- CACHED, addict, terminaltables: MIT\n- MMDetection, mmcv: Apache 2.0\n- pycocotools: FreeBSD\n\n\n### NON-COMMERCIAL codes removed from mmcv (Version 4)\n\nAlthough most of the codes are MIT or Apache, `mvcc` in the dependence contains a little code from NVIDIA StyleGAN. I have removed these NIVIDIA non-commercial codes. Check src/mmcv with `git status` and `git diff`.\n\nhttps://github.com/open-mmlab/mmcv/blob/main/LICENSES.md\n\n-> https://github.com/open-mmlab/mmcv/blob/2.x/mmcv/ops/csrc/pytorch/cuda/fused_bias_leakyrelu_cuda.cu\n\n-> https://nvlabs.github.io/stylegan2/license.html\n\n```\n3.3 Use Limitation. The Work and any derivative works thereof only may be used or intended for use non-commercially. \n```\n\nUnless these codes are removed or replaced, all object detection models in MM detection including CACHED, which depends on mmcv, cannot be used commercially.","metadata":{}},{"cell_type":"markdown","source":"## Use compiled packages","metadata":{}},{"cell_type":"code","source":"! pip install --no-index --find-links=file:///kaggle/input/cached \\\n  addict terminaltables pycocotools mmcv_full mmdet","metadata":{"execution":{"iopub.status.busy":"2023-06-20T04:01:19.183482Z","iopub.execute_input":"2023-06-20T04:01:19.184162Z","iopub.status.idle":"2023-06-20T04:01:30.202425Z","shell.execute_reply.started":"2023-06-20T04:01:19.184120Z","shell.execute_reply":"2023-06-20T04:01:30.201260Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import glob\nfrom mmdet.apis import init_detector, inference_detector  # depend heavily on mmcv\nfrom IPython.display import Image","metadata":{"execution":{"iopub.status.busy":"2023-06-20T04:01:30.206632Z","iopub.execute_input":"2023-06-20T04:01:30.206962Z","iopub.status.idle":"2023-06-20T04:01:35.693376Z","shell.execute_reply.started":"2023-06-20T04:01:30.206932Z","shell.execute_reply":"2023-06-20T04:01:35.692296Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"wdir = '/kaggle/input/cached/work_dirs'\nconfig_file = wdir + '/cascade_rcnn_swin-t_fpn_LGF_VCE_PCE_coco_focalsmoothloss/cascade_rcnn_swin-t_fpn_LGF_VCE_PCE_coco_focalsmoothloss.py'\ncheckpoint_file = wdir + '/cascade_rcnn_swin-t_fpn_LGF_VCE_PCE_coco_focalsmoothloss/checkpoint.pth'\n\nmodel = init_detector(config_file, checkpoint_file, device='cuda')\n\n\ndi = '/kaggle/input/benetech-making-graphs-accessible'\nfilenames = glob.glob(di + '/train/images/*.jpg')\nfilenames.sort()\n\nfor filename in filenames:\n    result = inference_detector(model, filename)  # list[array]\n    break","metadata":{"execution":{"iopub.status.busy":"2023-06-20T04:01:35.694799Z","iopub.execute_input":"2023-06-20T04:01:35.695184Z","iopub.status.idle":"2023-06-20T04:01:57.999496Z","shell.execute_reply.started":"2023-06-20T04:01:35.695149Z","shell.execute_reply":"2023-06-20T04:01:57.998502Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"`result` is a list of arrays for various classes (title, axis labels, ticke labels, ...), and the array has 5 elements for the bounding box coordinates and the confidence in [0, 1].","metadata":{}},{"cell_type":"code","source":"model.show_result(filename, result, out_file='./sample_result.jpg')\n\nImage('sample_result.jpg')","metadata":{"execution":{"iopub.status.busy":"2023-06-20T04:01:58.001874Z","iopub.execute_input":"2023-06-20T04:01:58.004741Z","iopub.status.idle":"2023-06-20T04:01:58.171074Z","shell.execute_reply.started":"2023-06-20T04:01:58.004702Z","shell.execute_reply":"2023-06-20T04:01:58.170140Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"##  Create wheel packages\n\nThe model is build upon mmcv (https://github.com/open-mmlab/mmcv), and it takes very long time to compile from source. I created the wheel packages and put them in kaggle datasets.\n\nThe following steps are not necessary once the wheel packages are created, but useful for different environment or CUDA versions.\n\n\n### mmcv\n\nhttps://github.com/open-mmlab/mmcv.git\n\n```\n! cp -r /kaggle/input/cached/src .\n%cd /kaggle/working/src/mmcv\n! CUDA_HOME=/usr/local/cuda-11.3 MMCV_WITH_OPS=1 pip wheel -w /kaggle/working .\n```\n\nBe patient and wait for 40 minutes with kaggle notebook. Important to specify the CUDA version same as PyTorch.\n\n### pycocotools\n\nSimilarly, download pycocotools and run `pip wheel`.\n\n```\n! pip donwload pycocotools  # => tar.gz source code\n````\n\nThis also contains CUDA codes and convinient to compile to whl file.\n\n### CACHED as mmdet\n\n```\ngit clone https://github.com/pengyu965/ChartDete\n```\n\nThis become the mmdet wheel package.\n","metadata":{}},{"cell_type":"code","source":"import torch\ntorch.version.cuda","metadata":{"execution":{"iopub.status.busy":"2023-06-20T04:01:58.172189Z","iopub.execute_input":"2023-06-20T04:01:58.172528Z","iopub.status.idle":"2023-06-20T04:01:58.179988Z","shell.execute_reply.started":"2023-06-20T04:01:58.172497Z","shell.execute_reply":"2023-06-20T04:01:58.178610Z"},"trusted":true},"execution_count":null,"outputs":[]}]}