{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":63134,"databundleVersionId":6880857,"sourceType":"competition"},{"sourceId":6887295,"sourceType":"datasetVersion","datasetId":3956417}],"dockerImageVersionId":30588,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Image Classification using FastAI\n\nIn this image classification project, we utilized the FastAI library to build and train a deep learning model for classifying images into different categories\n\nReference: https://docs.fast.ai/","metadata":{}},{"cell_type":"markdown","source":"## 1. Importing Libraries and Handling Truncated Images\n\n* ### Imported necessary libraries for image classification using FastAI.\n* ### Handled truncated images using the `ImageFile.LOAD_TRUNCATED_IMAGES` attribute.\n","metadata":{}},{"cell_type":"code","source":"from fastai.vision.all import *\nfrom fastai.vision.widgets import *\nfrom fastai.vision.augment import *","metadata":{"execution":{"iopub.status.busy":"2023-11-21T18:23:57.827625Z","iopub.execute_input":"2023-11-21T18:23:57.828592Z","iopub.status.idle":"2023-11-21T18:24:04.401056Z","shell.execute_reply.started":"2023-11-21T18:23:57.828556Z","shell.execute_reply":"2023-11-21T18:24:04.400120Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from PIL import ImageFile\nImageFile.LOAD_TRUNCATED_IMAGES = True","metadata":{"execution":{"iopub.status.busy":"2023-11-21T18:24:08.802068Z","iopub.execute_input":"2023-11-21T18:24:08.803016Z","iopub.status.idle":"2023-11-21T18:24:08.807000Z","shell.execute_reply.started":"2023-11-21T18:24:08.802982Z","shell.execute_reply":"2023-11-21T18:24:08.805940Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 2. Loading and Preprocessing Data\n\n* ### Defined the path to the training and test data.\n* ### Applied data augmentation transformations using the `Resize` function and using 'squish' method.\n* ### Created a `DataBlock` to load and preprocess the training data\n* ### Displayed a batch of training data using `show_batch()`","metadata":{}},{"cell_type":"code","source":"path = Path('/kaggle/input/anyas-gojo-revival-aerospace-enigma/train')","metadata":{"execution":{"iopub.status.busy":"2023-11-21T18:28:03.337259Z","iopub.execute_input":"2023-11-21T18:28:03.337660Z","iopub.status.idle":"2023-11-21T18:28:03.342234Z","shell.execute_reply.started":"2023-11-21T18:28:03.337618Z","shell.execute_reply":"2023-11-21T18:28:03.341247Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train = DataBlock(\n    blocks=(ImageBlock, CategoryBlock),\n    get_items=get_image_files,\n    splitter=RandomSplitter(valid_pct=0.2, seed=42),\n    get_y=parent_label,\n    item_tfms=[Resize(224, method='squish')]\n)\n\ndataloaders = train.dataloaders(path)\n\n# Show a batch of data\ndataloaders.show_batch(max_n=5)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T18:34:53.557610Z","iopub.execute_input":"2023-11-21T18:34:53.558049Z","iopub.status.idle":"2023-11-21T18:34:55.349431Z","shell.execute_reply.started":"2023-11-21T18:34:53.558019Z","shell.execute_reply":"2023-11-21T18:34:55.348525Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 3. Model Training\n\n* ### Created a vision learner using the `vision_learner` function with a ResNet50 architecture.\n* ### Fine-tuned the model for 10 epochs using the `fine_tune` method.\n* ### Monitored training progress with epoch-wise statistics.","metadata":{}},{"cell_type":"code","source":"learn = vision_learner(dataloaders, resnet50, metrics=accuracy)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T18:35:06.742532Z","iopub.execute_input":"2023-11-21T18:35:06.742910Z","iopub.status.idle":"2023-11-21T18:35:08.749743Z","shell.execute_reply.started":"2023-11-21T18:35:06.742880Z","shell.execute_reply":"2023-11-21T18:35:08.748810Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"learn.fine_tune(epochs=10)","metadata":{"execution":{"iopub.status.busy":"2023-11-20T12:15:50.813978Z","iopub.execute_input":"2023-11-20T12:15:50.814868Z","iopub.status.idle":"2023-11-20T12:18:46.709209Z","shell.execute_reply.started":"2023-11-20T12:15:50.814823Z","shell.execute_reply":"2023-11-20T12:18:46.708084Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 4. Making Predictions on Test Data\n\n* ### Loaded and the test data.\n* ### Made predictions on test images using the trained model.\n","metadata":{}},{"cell_type":"code","source":"test_path = Path('/kaggle/input/anyas-gojo-revival-aerospace-enigma/test')\nxtest = np.array(os.listdir(test_path))","metadata":{"execution":{"iopub.status.busy":"2023-11-20T12:19:08.600185Z","iopub.execute_input":"2023-11-20T12:19:08.600673Z","iopub.status.idle":"2023-11-20T12:19:08.607980Z","shell.execute_reply.started":"2023-11-20T12:19:08.600626Z","shell.execute_reply":"2023-11-20T12:19:08.607003Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ytest = []\nfor img in xtest:\n    with learn.no_bar(), learn.no_logging():\n        ytest.append(learn.predict(test_path/img)[0])\nytest = np.array(ytest)","metadata":{"execution":{"iopub.status.busy":"2023-11-20T12:19:32.047302Z","iopub.execute_input":"2023-11-20T12:19:32.047694Z","iopub.status.idle":"2023-11-20T12:19:43.267707Z","shell.execute_reply.started":"2023-11-20T12:19:32.047660Z","shell.execute_reply":"2023-11-20T12:19:43.266768Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"image_id = np.array([im_id.strip('.jpg') for im_id in xtest])","metadata":{"execution":{"iopub.status.busy":"2023-11-18T11:25:24.914171Z","iopub.execute_input":"2023-11-18T11:25:24.915021Z","iopub.status.idle":"2023-11-18T11:25:24.919652Z","shell.execute_reply.started":"2023-11-18T11:25:24.914986Z","shell.execute_reply":"2023-11-18T11:25:24.918709Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 5. Submission Preparation\n\n* ### Created a DataFrame with image paths and predicted classes.\n* ### Performed exploratory data analysis (EDA) on the submission DataFrame, including summary statistics and class distribution visualization.","metadata":{}},{"cell_type":"code","source":"submission = pd.DataFrame(data=[image_id, ytest]).T","metadata":{"execution":{"iopub.status.busy":"2023-11-18T11:37:32.220499Z","iopub.execute_input":"2023-11-18T11:37:32.220862Z","iopub.status.idle":"2023-11-18T11:37:32.237210Z","shell.execute_reply.started":"2023-11-18T11:37:32.220834Z","shell.execute_reply":"2023-11-18T11:37:32.236260Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission.columns = ['image_path', 'class']\n\nsubmission.head(4)","metadata":{"execution":{"iopub.status.busy":"2023-11-18T11:37:33.365060Z","iopub.execute_input":"2023-11-18T11:37:33.365425Z","iopub.status.idle":"2023-11-18T11:37:33.375102Z","shell.execute_reply.started":"2023-11-18T11:37:33.365392Z","shell.execute_reply":"2023-11-18T11:37:33.374254Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Performing EDA on the submission DataFrame\n# Displaying top rows\nprint(submission.head())\n\n# Summary statistics\nprint(submission.describe())\n","metadata":{"execution":{"iopub.status.busy":"2023-11-18T14:22:54.688663Z","iopub.execute_input":"2023-11-18T14:22:54.689710Z","iopub.status.idle":"2023-11-18T14:27:58.752101Z","shell.execute_reply.started":"2023-11-18T14:22:54.689665Z","shell.execute_reply":"2023-11-18T14:27:58.750650Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission[\"class\"]","metadata":{"execution":{"iopub.status.busy":"2023-11-18T11:37:36.445040Z","iopub.execute_input":"2023-11-18T11:37:36.445884Z","iopub.status.idle":"2023-11-18T11:37:36.454429Z","shell.execute_reply.started":"2023-11-18T11:37:36.445840Z","shell.execute_reply":"2023-11-18T11:37:36.453425Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission['Image_Number'] = submission['image_path'].str.extract('(\\d+)').astype(int)\n\n# Sort the DataFrame by the 'Image_Number' column\nsorted_df = submission.sort_values(by='Image_Number')\n\nprint(sorted_df)\n","metadata":{"execution":{"iopub.status.busy":"2023-11-18T11:37:40.360749Z","iopub.execute_input":"2023-11-18T11:37:40.361118Z","iopub.status.idle":"2023-11-18T11:37:40.376405Z","shell.execute_reply.started":"2023-11-18T11:37:40.361086Z","shell.execute_reply":"2023-11-18T11:37:40.375198Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sorted_df.drop(['Image_Number'],axis=1,inplace=True)","metadata":{"execution":{"iopub.status.busy":"2023-11-18T11:38:50.157845Z","iopub.execute_input":"2023-11-18T11:38:50.158295Z","iopub.status.idle":"2023-11-18T11:38:50.167923Z","shell.execute_reply.started":"2023-11-18T11:38:50.158256Z","shell.execute_reply":"2023-11-18T11:38:50.166664Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sorted_df","metadata":{"execution":{"iopub.status.busy":"2023-11-18T11:39:01.005881Z","iopub.execute_input":"2023-11-18T11:39:01.006297Z","iopub.status.idle":"2023-11-18T11:39:01.018183Z","shell.execute_reply.started":"2023-11-18T11:39:01.006254Z","shell.execute_reply":"2023-11-18T11:39:01.017287Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 6. Generating and Saving Submission\n\n* ### Generated a CSV file containing image paths and corresponding predicted classes.\n* ### Saved the submission DataFrame to a CSV file named 'submission1.csv'.\n","metadata":{}},{"cell_type":"code","source":"sorted_df.to_csv('submission1.csv',index=False)","metadata":{"execution":{"iopub.status.busy":"2023-11-18T11:40:17.468606Z","iopub.execute_input":"2023-11-18T11:40:17.469000Z","iopub.status.idle":"2023-11-18T11:40:17.476334Z","shell.execute_reply.started":"2023-11-18T11:40:17.468968Z","shell.execute_reply":"2023-11-18T11:40:17.475359Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Conclusion\nIn summary, the code demonstrates the end-to-end process of building an image classification model using FastAI, from data loading and preprocessing to model training, prediction, and submission preparation. The EDA provides a quick overview of the model's predictions and their distribution.\n","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}