{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Indroduction\n\nHello guys, I hope you had a chance to check out my EasyOCR finetuning . If you haven't, feel free to take a look and then come back to this [Notebook](https://www.kaggle.com/nadhirhasan/let-s-finetune-easyocr-boom). In this notebook, we'll be comparing the performance of the EasyOCR public model with our finetuned model. Let's dive in!","metadata":{}},{"cell_type":"code","source":"# Imports\nimport os\nimport pandas as pd\nimport cv2\nimport easyocr","metadata":{"execution":{"iopub.status.busy":"2023-05-11T14:38:34.378316Z","iopub.execute_input":"2023-05-11T14:38:34.379119Z","iopub.status.idle":"2023-05-11T14:38:36.392741Z","shell.execute_reply.started":"2023-05-11T14:38:34.379081Z","shell.execute_reply":"2023-05-11T14:38:36.391797Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"On the EasyOCR GitHub page, they explain how to access our custom pretrained model. To use our finetuned model, we need three files: the model architecture file (a .py file), a .yaml file containing network parameters and other information, and the finetuned model file (a .pth file). We should place the .py and .yaml files in the `.EasyOCR/user_network` folder and the .pth file in the `.EasyOCR/model` folder. It's important to name all three files the same.\n\nNow, let's take a look at how we can access our finetuned model.","metadata":{}},{"cell_type":"code","source":"# `I couldn't find the user_network folder in .EasyOCR, so I created one with that name. \n# We need this folder to store `.py` file and `.yaml` file.\n\n!mkdir /root/.EasyOCR/user_network","metadata":{"execution":{"iopub.status.busy":"2023-05-11T14:38:36.394987Z","iopub.execute_input":"2023-05-11T14:38:36.395664Z","iopub.status.idle":"2023-05-11T14:38:37.344141Z","shell.execute_reply.started":"2023-05-11T14:38:36.395629Z","shell.execute_reply":"2023-05-11T14:38:37.342905Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# copy-paste all three files (model architecture file, network parameter file, and finetuned model file) to the correct destination folders.\n\n!cp /kaggle/input/feasyocr-fintuned-model/best_accuracy.yaml /root/.EasyOCR/user_network/best_accuracy.yaml\n!cp /kaggle/input/feasyocr-fintuned-model/best_accuracy.py /root/.EasyOCR/user_network/best_accuracy.py\n!cp /kaggle/input/feasyocr-fintuned-model/best_accuracy.pth /root/.EasyOCR/model/best_accuracy.pth","metadata":{"execution":{"iopub.status.busy":"2023-05-11T14:38:37.347081Z","iopub.execute_input":"2023-05-11T14:38:37.348373Z","iopub.status.idle":"2023-05-11T14:38:40.298706Z","shell.execute_reply.started":"2023-05-11T14:38:37.348327Z","shell.execute_reply":"2023-05-11T14:38:40.297405Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"public_model_reader = easyocr.Reader(['en'],gpu=True)\nfinetuned_model_reader = easyocr.Reader(['en'], recog_network='best_accuracy',gpu=True) # In this code snippet, a parameter named `recog_network` is utilized and assigned the value of `best_accuracy`. This parameter is used to read three files that are all named `best_accuracy`, which is why they specified that these three files must have identical names.\n\n# Here, we are solely fine-tuning the recog_network of an OCR engine, OCR engine typically consists of two models: \n#  a detection model and a recognition model. The detection model's job is to identify the bounding boxes of each word, \n#  while the recognition model is responsible for recognizing the text within those boxes. \n#  However, in this particular case, we only fine-tunned the recognition model.","metadata":{"execution":{"iopub.status.busy":"2023-05-11T14:41:08.334682Z","iopub.execute_input":"2023-05-11T14:41:08.335449Z","iopub.status.idle":"2023-05-11T14:41:13.947584Z","shell.execute_reply.started":"2023-05-11T14:41:08.335412Z","shell.execute_reply":"2023-05-11T14:41:13.946602Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\n\nimport os\nimport pandas as pd\nimport cv2\nimport json\nimport matplotlib.pyplot as plt\nBASE_DIR = '/kaggle/input/benetech-making-graphs-accessible'\nfiles = os.listdir(f\"{BASE_DIR}/train/images\")\nf = open(f\"{BASE_DIR}/train/annotations/0000ae6cbdb1.json\")\nannotated_data = json.load(f)\n# annotated_data['text'][10]annotated_data['text'][10]\ndef hwlt2ltrb(coor):\n    left = coor[2]\n    top = coor[3]\n    right = left + coor[1]\n    bottom = top + coor[0]\n    \n    return (left,top,right,bottom)\n\n\ndef get_ltrb(source,idx):\n    l,t,r,b = float(\"inf\"),float(\"inf\"),float(\"-inf\"),float(\"-inf\")\n    data = source[idx][\"polygon\"]\n    x_values = (data['x0'],data['x1'],data['x2'],data['x3'])\n    y_values = (data['y0'],data['y1'],data['y2'],data['y3'])\n    l = min(l,min(x_values))\n    t = min(t,min(y_values))\n    r = max(r,max(x_values))\n    b = max(b,max(y_values))\n    text = source[idx]['text']\n    return (l,t,r,b),text\n\n\npublic_model_correct_count = 0\nfinetuned_model_correct_count = 0\n\nfor i,file in enumerate(files):\n    if i > 50000:\n        img = cv2.imread(f\"{BASE_DIR}/train/images/{file}\")\n        image = img.copy()\n        h,w,_ = img.shape\n        f = open(f\"{BASE_DIR}/train/annotations/{file.replace('jpg','json')}\")\n        annotated_data = json.load(f)\n\n        indicies = list(map(lambda x: x[\"id\"], annotated_data[\"axes\"][\"x-axis\"][\"ticks\"]))\n\n        indicies.extend(list(map(lambda x: x[\"id\"], annotated_data[\"axes\"][\"y-axis\"][\"ticks\"])))\n\n    #     plot_l,plot_t,plot_r,plot_b = hwlt2ltrb((plot_bb[\"height\"],plot_bb[\"width\"],plot_bb[\"x0\"],plot_bb[\"y0\"]))\n    #     try:\n        for idx in indicies:\n            try:\n                coor,text = get_ltrb(annotated_data['text'],idx)\n                l,t,r,b = coor\n                canvas = cv2.rectangle(image,(l,t),(r,b),(0,255,0),1)\n                text_crop = img[t:b,l:r]\n                img = text_crop\n                try:\n                    public_model_res = public_model_reader.readtext(img)\n                    if public_model_res[0][1] == text:\n                        public_model_correct_count += 1\n                except:\n                    pass\n\n                try:\n                    finetuned_model_res = finetuned_model_reader.readtext(img)\n\n                    if finetuned_model_res[0][1] == text:\n                        finetuned_model_correct_count += 1\n                except:\n                    pass\n                    \n            except:\n                pass\n            \n        if i % 1000 == 0:\n            print(f\"Exsist Model Per: {public_model_correct_count}\")\n            print(f\"Finetunned Model Per: {finetuned_model_correct_count}\")\n            print(\"#\" * 100)","metadata":{"execution":{"iopub.status.busy":"2023-05-11T14:41:15.986542Z","iopub.execute_input":"2023-05-11T14:41:15.987057Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Outro\n\nHere, it is evident that our fine-tuned model performs slightly better than the public model. However, there is still plenty of room for improvement. This is because the model was trained for only 100,000 iterations, which is why the performance is not optimal. I believe that better results can be achieved by tuning hyperparameters and increasing the iteration count. Unfortunately, I do not have access to a GPU of my own, which is why I was unable to train the model for more iterations. It would take a considerable amount of time. If you have access to a GPU, please feel free to train the model for more iterations and share your results with us.\n\n**Thanks For Reading**","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}