{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## Abstract\nBased on the recent 2 years competitions and discussions in kaggle community, this report is aimed at finding out the AI progress and achievement in image and text ares. According to the general summary based on competition titles, there are almost the one third of competitions related to image and video area. In general, this report analyzed the dicussion and metadata provided by Kaggle and combined with the external image AI research overview. In a result, the report summarize the following three main points. 1. Category and topic popularity summary for image/video AI in Kaggle community. 2. Common solutions and typical models used in image or video relate AI 3. Current image or video AI related applications and potential challenges. After the general analysis on writeup data and arxiv metadata, we found that dicussion writeup data could help us to get an overview about topic popularity and technique trends, while arxic metadata is more research focused and provides us a deeper insight on specific model development and progress. ","metadata":{}},{"cell_type":"markdown","source":"## 1. Category and topic popularity summary\n### Popularity comparison in general category\nAccording to the analysis on the write_up csv filer, there are 3127 records in total and 1149 records are related to image or video topic, which take up to 36%. According to the topic summary, there are 310 competitions in total while 171 are related to the visual competitions, more than half of the competitions in Kaggle for the last 2 year are about image/video. In general, the image/video related AI competition are the most popular topic in Kaggle AI competition.","metadata":{}},{"cell_type":"code","source":"import matplotlib.pyplot as plt\nlabels = 'Text', 'Image', 'Tabluar', 'Other'\nsizes = [46, 171, 25, 68]\n\nfig, ax = plt.subplots()\n_ = ax.pie(sizes, labels=labels,\n       colors=['#D4C2AD', '#D7A184', '#EFDFCC', '#66796B'])","metadata":{"execution":{"iopub.status.busy":"2023-07-12T00:15:04.910866Z","iopub.execute_input":"2023-07-12T00:15:04.912101Z","iopub.status.idle":"2023-07-12T00:15:05.184168Z","shell.execute_reply.started":"2023-07-12T00:15:04.912047Z","shell.execute_reply":"2023-07-12T00:15:05.182033Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### What do users find in certain areas? (Medical / Health care / Geography)\nAfter extracting all the titles related to the imgae/video competitions, we're trying to figure out what area do people usually interested in image/video related AI techniques. After summarize the top 5 most frequent talked competitions, the result shows that as belows.\n\n- Human Protein Atlas - Single Cell Classification 30\n- Cassava Leaf Disease Classification 28\n- Shopee - Price Match Guarantee 27\n- APTOS 2019 Blindness Detection 27\n- RANZCR CLiP - Catheter and Line Position Challenge 24\n\nThe result indicates that most image/video AI technique are focused in medical area. Next is business and commercial and statellite and geography comes the third. However, from the wordnet auto generated image, it's hard to see if there is any other connections between models and research area. It's also hard to tell what's the most common used model in image AI area. Thus, we need to take a closer look to dig into the discussion board. Or we need more supplement analysis from archived metadata file.\n\n","metadata":{}},{"cell_type":"markdown","source":"## 1.1 Topic distribution in image and video AI\n#### Based on deep learning models\n- Image recognition\n- Image classification\n\n[Deep Learning Model Topic Distribution from write_up file](#dp_model_topic_distribution)","metadata":{}},{"cell_type":"markdown","source":"Image recognition by computer is usually divided into two major steps: image feature extraction and image classification prediction. Firstly, the input image is is preprocessed and processed into a form suitable for feature extraction, then the features of the image are extracted, after which the feature image is classified and predicted. After that, the feature images are classified and predicted,Among them, image preprocessing is mainly for the purpose of eliminating interference, enhancing target image information, and better performing image feature extraction.","metadata":{}},{"cell_type":"markdown","source":"#### Based on research area\n- Medical area\n- Business and commercial\n- Geographic detection\n- Astronomy\n\nBased on the ranking result for most frequently discussed competitions, we could see medical area, like Human Protein Atlas, APTOS 2019 Blindness Detection are in need of image and video related AI techniques to help with disease detection and medical treatment. Other than that, according to the category ranking result from arxic metadata, we found that astronomy and geographic area also contributed lots of image and video related AI progress.\n\n[Categoty Distribution from arxiv_metadata](#category_distribution)","metadata":{}},{"cell_type":"markdown","source":"## 2. Common solutions and models\n### Convolutional Neural Networks (CNN)\nIn general, deep learning can be divided into supervised learning and unsupervised learning on the basis of whether the data contains .In the process of supervised learning, the mapping relationship between the marks and the features of the training data will be found, and the deviations in the learning process will be continuously corrected through the marks, and the prediction rate of learning will be continuously improved. Supervised learning mainly includes CNN, Recurrent Neural Networks (RNN) and Deep Stacked Networks (DSN).The training data of unsupervised learning does not have marks. Commonly used algorithms include Restricted Boltzmann Machine (RBM), and Deep Belief Network (DBN). Image data can be considered as a set of two-dimensional pixel value input. Convolutional Neural Networks (CNN) is composed of several convolution layers and fully connected layers and includes related weights and pooling layers. Such structure enables CNN to make full use of the two- dimensional structure of the data, so that the recognition image has a higher accuracy rate. CNN also uses standard backpropagation algorithm for training. With fewer parameter estimates, it is easier to train than other deep models. Figure 2 shows the basic model of image recognition based on CNN. First, the model divides the input image by region, then enters the image into CNN network for feature extraction and recognition, and finally classifies the image based on image features.\n\n[WordCloud based on Metadata Title Summary](#word_cloud_pic)","metadata":{}},{"cell_type":"markdown","source":"### Common preprocessing methods\n#### Graying\nChanging RGB image to gray image to mainly using component method and weighted average method\n\n#### Geometric transformation\nMainly using interpolation method to make spatial transformation on the image space to resuce image error information\n\n#### Image enhancement\nEnhancing the target image infortion, including grayscale transformation, histogram correction and filtering methods. Traditional image recognition methods need to manually extract image features, and the recognition accurace depends on the accuracy of feature extraction.\n","metadata":{}},{"cell_type":"markdown","source":"## 3. Applications in image related AI\n#### Medical image recognition\nMedical images have the disadvantages of wide variety, low resolution, and heavy dependence on imaging equipment and imaging environment, which to a certain extent limit doctors to make effective diagnosis of patients' symptoms. The use of image processing technology combined with deep learning to diagnose the diseased parts of the human body is one of the most cutting-edge medical diagnosis methods today. In recent years, the rapid development of deep learning models, especially the widespread use of CNN, has made it a mainstream trend to use neural network models to automatically extract and select features and perform classification. different variants of CNN models have been widely used in clinical disease diagnosis based on medical images, for example, based on Kaggle's public dataset of fundus images, Shanthi T et al. used improved AlexNet for the classification of diabetic retinopathy, and its accuracy can reach about 96.6%; based on VGG, the accuracy of benign and malignant classification of lung nodules using chest films can be as high as 99%. Currently, among the common CNN variants, ResNet and VGG have the best performance in medical image classification, so most of the tumor detection, brain neurological disease classification, and cardiovascular disease detection use these two models as baseline models for research.\n\n#### Face recognition\nFace recognition is one of the important applications of computer vision, and face recognition in a broad sense includes image acquisition, face detection, face alignment, feature representation, etc. Face recognition is widely used in many fields such as security, finance and e-government, etc. Improving the performance of face recognition is very important for expanding the application field and practicality of face recognition. Traditional face recognition is limited in its accuracy due to factors such as illumination, pose and expression changes, occlusion, massive data and its own limitations. The emergence of deep learning has led to a breakthrough in face recognition technology, where learning algorithms learn discriminative face features directly from the original image, making it a great improvement in speed and accuracy. In 2021, Shuaifeng Huang gave an improved face recognition method based on LeNet-5, in which the structure of a typical convolutional neural network LeNet-5 is designed to consist of two convolutional sampling layers, a fully connected hidden layer and a classification output layer, which reduces the complexity of the network structure. In addition, the number of convolutional kernels is reduced, the pooling method and the classification output method are improved, and the computational complexity is reduced. It is demonstrated that the method improves the speed of single face recognition on the embedded platform while ensuring the training and testing accuracy.","metadata":{}},{"cell_type":"markdown","source":"## 4. Stretch: image to text AI libraries\nBesides the progress in image classification and recognition, there is also a large progress in AI techniques about converting image and text and vice versa. According to Hugging Face and Replicate recommendation, as long as the extracted keywords from writeup file, here are some popular libraries users frequently used.\n\n- Optical Character Recognition (OCR)\n\nOCR models convert the text present in an image, e.g. a scanned document, to text.\n\n- Pix2Struct\n\nPix2Struct is a state-of-the-art model built and released by Google AI. The model itself has to be trained on a downstream task to be used. These tasks include, captioning UI components, images including text, visual questioning infographics, charts, scientific diagrams and more. You can find these models on recommended models of this page.\n\n- BLIP\n\nBootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation","metadata":{}},{"cell_type":"markdown","source":"## Reference\n+ Lu, Z.X., Qian, P., Bi, D., Ye, Z.W., He, X., Zhao, Y.H., Su, L., Li, S.L. and Zhu, Z.L., 2021. Application of AI and IoT in clinical medicine: summary and challenges. Current medical science, 41, pp.1134-1150.\n+ Zhangqi, Zhang Rongmei, Chen bin. A Review of Image Recognition Technology Research Based on Deep Learning[J].Journal of the Hebei Academy of Sciences,2019,36(03):28-36\n+ Shao L, Wu D, Li X.Learning deep and wide:a spectral method for learning deep networks[J].IEEE Transactions on Neural Networks and Learning Systems, 2014, 25(12):2303-2308.\n+ Wang Lihui. Research progress and development trend of deep learning in medical imaging[J].Big Data Research.2020,6(06):83-104.","metadata":{}},{"cell_type":"markdown","source":"## Source Code\n### Analysis on Kaggle Write up dataset","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport csv\n\n# with open(\"/kaggle/input/2023-kaggle-ai-report/kaggle_writeups_20230510.csv\") as file:\n#   csv_reader = csv.reader(file)\n\nwrite_ups1 = pd.read_csv('/kaggle/input/2023-kaggle-ai-report/kaggle_writeups_20230510.csv', on_bad_lines='skip', engine='python')\nwrite_ups1.head()","metadata":{"execution":{"iopub.status.busy":"2023-07-12T04:52:01.858107Z","iopub.execute_input":"2023-07-12T04:52:01.858696Z","iopub.status.idle":"2023-07-12T04:52:02.297582Z","shell.execute_reply.started":"2023-07-12T04:52:01.858671Z","shell.execute_reply":"2023-07-12T04:52:02.296656Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"write_ups1['Title of Competition'].nunique()","metadata":{"execution":{"iopub.status.busy":"2023-07-12T04:52:21.671157Z","iopub.execute_input":"2023-07-12T04:52:21.671493Z","iopub.status.idle":"2023-07-12T04:52:21.682358Z","shell.execute_reply.started":"2023-07-12T04:52:21.671465Z","shell.execute_reply":"2023-07-12T04:52:21.681328Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Basic analysis on research categories\n# language related competition counts\nlanguage_comps = write_ups1[(write_ups1.Writeup.str.contains(\n    'language', 'text') | write_ups1['Title of Competition'].str.contains(\n    'language')).fillna(False)]['Title of Competition'].unique()\nprint(f\"How many now: {len(language_comps)}\")\nlanguage_comps[:5]","metadata":{"execution":{"iopub.status.busy":"2023-07-12T00:23:47.924416Z","iopub.execute_input":"2023-07-12T00:23:47.925338Z","iopub.status.idle":"2023-07-12T00:23:47.976872Z","shell.execute_reply.started":"2023-07-12T00:23:47.925283Z","shell.execute_reply":"2023-07-12T00:23:47.975446Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# image related competition counts\nimage_comps = write_ups1[(write_ups1.Writeup.str.contains(\n    'image', 'video') | write_ups1['Title of Competition'].str.contains(\n    'image', 'video')).fillna(False)]['Title of Competition'].unique()\nprint(f\"How many now: {len(image_comps)}\")\nimage_comps[:5]","metadata":{"execution":{"iopub.status.busy":"2023-07-12T04:52:26.303521Z","iopub.execute_input":"2023-07-12T04:52:26.303966Z","iopub.status.idle":"2023-07-12T04:52:26.337430Z","shell.execute_reply.started":"2023-07-12T04:52:26.303932Z","shell.execute_reply":"2023-07-12T04:52:26.336416Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# image related competition counts\ntabular_comps = write_ups1[(write_ups1.Writeup.str.contains(\n    'tabular') | write_ups1['Title of Competition'].str.contains(\n    'tabular')).fillna(False)]['Title of Competition'].unique()\nprint(f\"How many now: {len(tabular_comps)}\")\ntabular_comps[:5]","metadata":{"execution":{"iopub.status.busy":"2023-07-12T00:23:54.726687Z","iopub.execute_input":"2023-07-12T00:23:54.727759Z","iopub.status.idle":"2023-07-12T00:23:54.778368Z","shell.execute_reply.started":"2023-07-12T00:23:54.727704Z","shell.execute_reply":"2023-07-12T00:23:54.777073Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%capture\n!pip install dataprep","metadata":{"scrolled":true,"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-07-12T04:52:43.095052Z","iopub.execute_input":"2023-07-12T04:52:43.095472Z","iopub.status.idle":"2023-07-12T04:53:13.816583Z","shell.execute_reply.started":"2023-07-12T04:52:43.095430Z","shell.execute_reply":"2023-07-12T04:53:13.815562Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Start with loading all necessary libraries\nimport numpy as np\nimport pandas as pd\nfrom os import path\nfrom PIL import Image\nfrom wordcloud import WordCloud, STOPWORDS, ImageColorGenerator\nimport matplotlib.pyplot as plt","metadata":{"execution":{"iopub.status.busy":"2023-07-12T04:52:13.469390Z","iopub.execute_input":"2023-07-12T04:52:13.469728Z","iopub.status.idle":"2023-07-12T04:52:13.543207Z","shell.execute_reply.started":"2023-07-12T04:52:13.469702Z","shell.execute_reply":"2023-07-12T04:52:13.541741Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import matplotlib.pyplot as plt\nlabels = 'Text', 'Image', 'Tabluar', 'Other'\nsizes = [46, 171, 25, 68]\n\nfig, ax = plt.subplots()\n_ = ax.pie(sizes, labels=labels,\n       colors=['#D4C2AD', '#D7A184', '#EFDFCC', '#66796B'])","metadata":{"execution":{"iopub.status.busy":"2023-07-12T00:15:05.752768Z","iopub.status.idle":"2023-07-12T00:15:05.753724Z","shell.execute_reply.started":"2023-07-12T00:15:05.753423Z","shell.execute_reply":"2023-07-12T00:15:05.753451Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Extract image related competition and discussion data","metadata":{}},{"cell_type":"code","source":"from dataprep.clean import clean_headers\n# filter the result only related to image or video\nvision_comps = write_ups1[(write_ups1.Writeup.str.contains(\n    'image', 'video') | write_ups1['Title of Competition'].str.contains(\n    'image', 'video')).fillna(False)]\nvision_comps.head()\nprint(len(vision_comps))\nvision_df = clean_headers(vision_comps)\nimage_dis = vision_df[\"title_of_competition\"].value_counts().to_frame()\nprint(image_dis.head())","metadata":{"execution":{"iopub.status.busy":"2023-07-12T04:54:50.367479Z","iopub.execute_input":"2023-07-12T04:54:50.367809Z","iopub.status.idle":"2023-07-12T04:54:51.537547Z","shell.execute_reply.started":"2023-07-12T04:54:50.367779Z","shell.execute_reply":"2023-07-12T04:54:51.536495Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Deep learning model in image related AI topic distribution\n<a id=\"dp_model_topic_distribution\"></a>","metadata":{}},{"cell_type":"code","source":"# generate distribution pie chart for image related AI writeups\nvision_comps_classification = write_ups1[(write_ups1.Writeup.str.contains(\n    'classification') | write_ups1['Title of Competition'].str.contains(\n    'classification')).fillna(False)]\nvision_comps_recognition = write_ups1[(write_ups1.Writeup.str.contains(\n    'recognition') | write_ups1['Title of Competition'].str.contains(\n    'recognition')).fillna(False)]\nprint(\"classification related count:\", len(vision_comps_classification))\nprint(\"recognition related count:\", len(vision_comps_recognition))\n\nimport matplotlib.pyplot as plt\nlabels = 'Classification', 'Recognition', 'Other'\nsizes = [len(vision_comps_classification), len(vision_comps_recognition), 1149 - len(vision_comps_recognition) - len(vision_comps_classification)]\n\nfig, ax = plt.subplots()\n_ = ax.pie(sizes, labels=labels,\n       colors=['#D4C2AD', '#D7A184', '#66796B'])","metadata":{"execution":{"iopub.status.busy":"2023-07-12T00:15:05.758171Z","iopub.status.idle":"2023-07-12T00:15:05.759172Z","shell.execute_reply.started":"2023-07-12T00:15:05.758863Z","shell.execute_reply":"2023-07-12T00:15:05.758892Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Top 10 of most frequently discussed competitions in image related AI","metadata":{}},{"cell_type":"code","source":"# Using groupby() and count()\ndf2 = vision_df.groupby(['title_of_competition'])['title_of_competition'].count().reset_index(name='counts')\nsortedDF=df2.sort_values('counts', ascending=False)\n\n# select top 10 to visualize\ndf2 = sortedDF.iloc[:10]\nprint(df2)","metadata":{"execution":{"iopub.status.busy":"2023-07-12T05:01:33.117839Z","iopub.execute_input":"2023-07-12T05:01:33.118401Z","iopub.status.idle":"2023-07-12T05:01:33.128009Z","shell.execute_reply.started":"2023-07-12T05:01:33.118373Z","shell.execute_reply":"2023-07-12T05:01:33.126480Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y=df2['title_of_competition']\n \n# getting values against each value of y\nx=df2['counts']\nplt.barh(y, x, color ='#BA8E7A')\n \n# setting label of y-axis\nplt.ylabel(\"competition title\")\n \n# setting label of x-axis\nplt.xlabel(\"counts\")\nplt.title(\"Horizontal bar graph\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-07-12T05:01:49.601825Z","iopub.execute_input":"2023-07-12T05:01:49.602257Z","iopub.status.idle":"2023-07-12T05:01:50.031423Z","shell.execute_reply.started":"2023-07-12T05:01:49.602225Z","shell.execute_reply":"2023-07-12T05:01:50.030500Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Analysis on most frequently discussed competition","metadata":{}},{"cell_type":"code","source":"df = clean_headers(write_ups1)\nriid = df[(df['title_of_competition']==\"Human Protein Atlas - Single Cell Classification\")].reset_index(drop=True)\nnRow, nCol = riid.shape\nprint(f'There are {nRow} rows and {nCol} columns')\nriid.tail()","metadata":{"execution":{"iopub.status.busy":"2023-07-12T00:15:05.766684Z","iopub.status.idle":"2023-07-12T00:15:05.767652Z","shell.execute_reply.started":"2023-07-12T00:15:05.767349Z","shell.execute_reply":"2023-07-12T00:15:05.767388Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"text = \" \".join(titl for titl in riid.title_of_writeup)\n\nstopwords = set(STOPWORDS)\nstop_list = ['solution', 'place']\nstopwords = list(STOPWORDS) + stop_list","metadata":{"execution":{"iopub.status.busy":"2023-07-12T00:15:05.769363Z","iopub.status.idle":"2023-07-12T00:15:05.770354Z","shell.execute_reply.started":"2023-07-12T00:15:05.770057Z","shell.execute_reply":"2023-07-12T00:15:05.770085Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"wordcloud = WordCloud(stopwords=stopwords, max_font_size=50, max_words=500,\n                      width = 600, height = 400,\n                      colormap='Purples',\n                      background_color=\"#EFDFCC\").generate(text)\nplt.figure(figsize=(12,8))\nplt.imshow(wordcloud, interpolation=\"bilinear\")\nplt.axis(\"off\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-07-12T00:15:05.772155Z","iopub.status.idle":"2023-07-12T00:15:05.773152Z","shell.execute_reply.started":"2023-07-12T00:15:05.772857Z","shell.execute_reply":"2023-07-12T00:15:05.772886Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"As we can see from the image, most of the words are related to the rankings and after take a deeper look into the discussion board, I figured thatit's hard to extraxt useful information from write-up titles due to most of the titles are tend to attract viewers attention by using ranks or extravagant vocabularies. Therefore, I'm trying to analyze the content text to see if there is any useful information we can extract. Other that that, we could still see some of the quite popular discussion keywords like, 'Transformers\", \"CNN', \"Classifiers\". 'Multi-label'. From here, I raised another question, what do they talk about these keywords in particular? Like what transformer do they use in order to get higher rank? How do they design the transformers to be suitable for the problem? Therefore, we need to analyze context text data to get more specific information.","metadata":{}},{"cell_type":"code","source":"transformer_topic = riid[(riid.writeup.str.contains(\n    'CNN')).fillna(False)]","metadata":{"execution":{"iopub.status.busy":"2023-07-12T00:15:05.774799Z","iopub.status.idle":"2023-07-12T00:15:05.775632Z","shell.execute_reply.started":"2023-07-12T00:15:05.775344Z","shell.execute_reply":"2023-07-12T00:15:05.775369Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Analysis based on the discussion content\nimage_text = \" \".join(titl for titl in transformer_topic.writeup)\n\n# remove html tags\nimport re\nCLEANR = re.compile('<.*?>') \n\ndef cleanhtml(raw_html):\n  cleantext = re.sub(CLEANR, '', raw_html)\n  return cleantext\n\nimage_text = cleanhtml(image_text)","metadata":{"execution":{"iopub.status.busy":"2023-07-12T00:15:05.777080Z","iopub.status.idle":"2023-07-12T00:15:05.777917Z","shell.execute_reply.started":"2023-07-12T00:15:05.777630Z","shell.execute_reply":"2023-07-12T00:15:05.777656Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# write up visualization\nwordcloud = WordCloud(stopwords=stopwords, max_font_size=50, max_words=500,\n                      width = 600, height = 400,\n                      colormap='Purples',\n                      background_color=\"#D4C2AD\").generate(image_text)\nplt.figure(figsize=(12,8))\nplt.imshow(wordcloud, interpolation=\"bilinear\")\nplt.axis(\"off\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-07-12T00:15:05.779345Z","iopub.status.idle":"2023-07-12T00:15:05.780182Z","shell.execute_reply.started":"2023-07-12T00:15:05.779909Z","shell.execute_reply":"2023-07-12T00:15:05.779936Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"It's hard to get the conclusion from the word cloud picture directly. Therefore, at the end of this notebook, I decided to use an text AI model, given the all the write_up data and see what text model could summarize.","metadata":{}},{"cell_type":"code","source":"# Analysis based on the discussion content\nimage_text = \" \".join(titl for titl in riid.writeup)\n\n# remove html tags\nimport re\nCLEANR = re.compile('<.*?>') \n\ndef cleanhtml(raw_html):\n  cleantext = re.sub(CLEANR, '', raw_html)\n  return cleantext\n\nimage_text = cleanhtml(image_text)","metadata":{"execution":{"iopub.status.busy":"2023-07-12T00:15:05.781421Z","iopub.status.idle":"2023-07-12T00:15:05.782854Z","shell.execute_reply.started":"2023-07-12T00:15:05.782518Z","shell.execute_reply":"2023-07-12T00:15:05.782549Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# write up visualization\nwordcloud = WordCloud(stopwords=stopwords, max_font_size=50, max_words=500,\n                      width = 600, height = 400,\n                      colormap='Purples',\n                      background_color=\"#D4C2AD\").generate(image_text)\nplt.figure(figsize=(12,8))\nplt.imshow(wordcloud, interpolation=\"bilinear\")\nplt.axis(\"off\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-07-12T00:15:05.784121Z","iopub.status.idle":"2023-07-12T00:15:05.784692Z","shell.execute_reply.started":"2023-07-12T00:15:05.784408Z","shell.execute_reply":"2023-07-12T00:15:05.784434Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Analysis on arxiv metadata dataset","metadata":{}},{"cell_type":"code","source":"import json\nimport pandas as pd\n\n# with open('/kaggle/input/2023-kaggle-ai-report/arxiv_metadata_20230510.json', 'r') as f:\n#     data = json.load(f)\ndata = [json.loads(line)\n        for line in open('/kaggle/input/2023-kaggle-ai-report/arxiv_metadata_20230510.json', 'r', encoding='utf-8')]\n\ndf = pd.DataFrame.from_dict(data)","metadata":{"execution":{"iopub.status.busy":"2023-07-12T00:15:05.786730Z","iopub.status.idle":"2023-07-12T00:15:05.787847Z","shell.execute_reply.started":"2023-07-12T00:15:05.787412Z","shell.execute_reply":"2023-07-12T00:15:05.787440Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Extract image related titles for future analysis\nimage_metadata = df[(df['abstract'].str.contains(\n    'image') | df['title'].str.contains(\n    'image')).fillna(False)]\nprint(len(image_metadata))","metadata":{"execution":{"iopub.status.busy":"2023-07-12T00:15:05.789150Z","iopub.status.idle":"2023-07-12T00:15:05.789816Z","shell.execute_reply.started":"2023-07-12T00:15:05.789464Z","shell.execute_reply":"2023-07-12T00:15:05.789494Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Extract data related to classification only\nimage_classification_metadata = image_metadata[(image_metadata['abstract'].str.contains(\n    'classification') | image_metadata['title'].str.contains(\n    'classification')).fillna(False)]\nprint(len(image_classification_metadata))","metadata":{"execution":{"iopub.status.busy":"2023-07-12T00:15:05.792918Z","iopub.status.idle":"2023-07-12T00:15:05.793498Z","shell.execute_reply.started":"2023-07-12T00:15:05.793209Z","shell.execute_reply":"2023-07-12T00:15:05.793235Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"text = \" \".join(titl for titl in image_classification_metadata.title)","metadata":{"execution":{"iopub.status.busy":"2023-07-12T00:15:05.795649Z","iopub.status.idle":"2023-07-12T00:15:05.796271Z","shell.execute_reply.started":"2023-07-12T00:15:05.795974Z","shell.execute_reply":"2023-07-12T00:15:05.796003Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## WordCloud based on Metadata Title Summary\n<a id=\"word_cloud_pic\"></a>","metadata":{}},{"cell_type":"code","source":"stopwords = set(STOPWORDS)\nstop_list = ['Classification', 'Deep Learning']\nstopwords = list(STOPWORDS) + stop_list\n\nwordcloud = WordCloud(stopwords=stopwords, max_font_size=50, max_words=500,\n                      width = 600, height = 400,\n                      colormap='Purples',\n                      background_color=\"orange\").generate(text)\nplt.figure(figsize=(12,8))\nplt.imshow(wordcloud, interpolation=\"bilinear\")\nplt.axis(\"off\")\nplt.show()\nplt.savefig('sample.png')","metadata":{"execution":{"iopub.status.busy":"2023-07-12T00:15:05.798167Z","iopub.status.idle":"2023-07-12T00:15:05.798794Z","shell.execute_reply.started":"2023-07-12T00:15:05.798468Z","shell.execute_reply":"2023-07-12T00:15:05.798495Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Extract data related to recognition only\nimage_recognition_metadata = image_metadata[(image_metadata['abstract'].str.contains(\n    'recognition') | image_metadata['title'].str.contains(\n    'recognition')).fillna(False)]\nprint(len(image_recognition_metadata))","metadata":{"execution":{"iopub.status.busy":"2023-07-12T00:15:05.800251Z","iopub.status.idle":"2023-07-12T00:15:05.800838Z","shell.execute_reply.started":"2023-07-12T00:15:05.800536Z","shell.execute_reply":"2023-07-12T00:15:05.800564Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"text1 = \" \".join(titl for titl in image_recognition_metadata.title)","metadata":{"execution":{"iopub.status.busy":"2023-07-12T00:15:05.802544Z","iopub.status.idle":"2023-07-12T00:15:05.803324Z","shell.execute_reply.started":"2023-07-12T00:15:05.802913Z","shell.execute_reply":"2023-07-12T00:15:05.802946Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"stopwords = set(STOPWORDS)\nstop_list = ['recognition', 'Deep Learning', 'Neural']\nstopwords = list(STOPWORDS) + stop_list\n\nwordcloud1 = WordCloud(stopwords=stopwords, max_font_size=50, max_words=500,\n                      width = 600, height = 400,\n                      colormap='Purples',\n                      background_color=\"orange\").generate(text1)\nplt.figure(figsize=(12,8))\nplt.imshow(wordcloud, interpolation=\"bilinear\")\nplt.axis(\"off\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-07-12T00:15:05.806799Z","iopub.status.idle":"2023-07-12T00:15:05.807647Z","shell.execute_reply.started":"2023-07-12T00:15:05.807315Z","shell.execute_reply":"2023-07-12T00:15:05.807346Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Extract the neural network related papers\n# Extract data related to classification only\nimage_nn_metadata = image_metadata[(image_metadata['abstract'].str.contains(\n    'neural network') | image_metadata['title'].str.contains(\n    'neural network')).fillna(False)]\nprint(len(image_nn_metadata))","metadata":{"execution":{"iopub.status.busy":"2023-07-12T00:15:05.809257Z","iopub.status.idle":"2023-07-12T00:15:05.809852Z","shell.execute_reply.started":"2023-07-12T00:15:05.809555Z","shell.execute_reply":"2023-07-12T00:15:05.809581Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"text2 = \" \".join(titl for titl in image_nn_metadata.title)","metadata":{"execution":{"iopub.status.busy":"2023-07-12T00:15:05.811248Z","iopub.status.idle":"2023-07-12T00:15:05.811796Z","shell.execute_reply.started":"2023-07-12T00:15:05.811530Z","shell.execute_reply":"2023-07-12T00:15:05.811555Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"stopwords = set(STOPWORDS)\nstop_list = ['Neural Network', 'Deep Learning', 'Image']\nstopwords = list(STOPWORDS) + stop_list\n\nwordcloud = WordCloud(stopwords=stopwords, max_font_size=50, max_words=500,\n                      width = 600, height = 400,\n                      colormap='Purples',\n                      background_color=\"orange\").generate(text2)\nplt.figure(figsize=(12,8))\nplt.imshow(wordcloud, interpolation=\"bilinear\")\nplt.axis(\"off\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-07-12T00:15:05.813696Z","iopub.status.idle":"2023-07-12T00:15:05.814291Z","shell.execute_reply.started":"2023-07-12T00:15:05.813996Z","shell.execute_reply":"2023-07-12T00:15:05.814022Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Since it's hard to get any conclusion directly from the word cloud image, I decide to rank the papers according to their comments index and select top 5 papers for more details.","metadata":{}},{"cell_type":"markdown","source":"### AI Research Category Ranking\n<a id='category_distribution'></a>","metadata":{}},{"cell_type":"code","source":"image_metadata['categories'].nunique()","metadata":{"execution":{"iopub.status.busy":"2023-07-12T00:15:05.815688Z","iopub.status.idle":"2023-07-12T00:15:05.816259Z","shell.execute_reply.started":"2023-07-12T00:15:05.815964Z","shell.execute_reply":"2023-07-12T00:15:05.815989Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Using groupby() and count()\ndf2 = image_metadata.groupby(['categories'])['categories'].count().reset_index(name='counts')\nsortedDF=df2.sort_values('counts', ascending=False)\n\n# select top 10 to visualize\ndf3 = sortedDF.iloc[:20]\nprint(df3)# Using groupby() and count()\ndf2 = df.groupby(['categories'])['categories'].count().reset_index(name='counts')\nsortedDF=df2.sort_values('counts', ascending=False)\n\n# select top 10 to visualize\ndf2 = sortedDF.iloc[:20]\nprint(df2)","metadata":{"execution":{"iopub.status.busy":"2023-07-12T00:15:05.818477Z","iopub.status.idle":"2023-07-12T00:15:05.819046Z","shell.execute_reply.started":"2023-07-12T00:15:05.818756Z","shell.execute_reply":"2023-07-12T00:15:05.818781Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Using groupby() and count()\n#df2 = image_metadata.groupby(['categories'])['categories'].count().reset_index(name='counts')\nsortedDF2=image_metadata.sort_values('comments', ascending=True)\n\n# select top 10 to visualize\ndf4 = sortedDF2.iloc[:20]\nprint(df4['title'])","metadata":{"execution":{"iopub.status.busy":"2023-07-12T00:15:05.820333Z","iopub.status.idle":"2023-07-12T00:15:05.820888Z","shell.execute_reply.started":"2023-07-12T00:15:05.820601Z","shell.execute_reply":"2023-07-12T00:15:05.820627Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#export csv file\nexport_file = {'type': ['essay_category', 'essay_url', 'feedback1_url', 'feedback2_url', 'feedback3_url'],\n              'value':['image and video', 'https://www.kaggle.com/code/sabrinahan/kaggle-image-and-video-related-ai-report', \n                       'https://www.kaggle.com/code/sabrinahan/kaggle-image-and-video-related-ai-report/comments', \n                       'https://mayuri7.notion.site/mayuri7/sarah-advice-1bc9a116e4574819a7f9aca917d09428', \n                       '_3']}\n\nexport_df = pd.DataFrame(export_file)\n\nexport_df.to_csv('submission.csv', index=False, header=True)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Adventure: How would Dolly evaluate AI progress?\n\n### Answer from a ChatGPT bot","metadata":{}},{"cell_type":"code","source":"ans = \"\"\"\n{\n    \"id\": \"chatcmpl-JN8VtbBr8PpujAWWDAWNat\",\n    \"object\": \"chat.completion\",\n    \"created\": 1689252444,\n    \"model\": \"dolly-v2-7b\",\n    \"choices\": [\n        {\n            \"index\": 0,\n            \"message\": {\n                \"role\": \"assistant\",\n                \"content\": \"Kaggle is doing with image and video related AI, they provide a platform for data scientists and engineers to share their machine learning code, data, and models, and invite other participants to collaborate and compete on large data sets to improve and validate those models. And Kaggle has a large number of competitions for data scientists to sharpen their skills and for enthusiasts to have fun.\\n\\n\"\n            },\n            \"finish_reason\": \"stop\"\n        }\n    ],\n    \"usage\": {\n        \"prompt_tokens\": 37,\n        \"total_tokens\": 115,\n        \"completion_tokens\": 78\n    }\n}\n\n\"\"\"","metadata":{"scrolled":true,"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-07-12T15:57:53.802394Z","iopub.execute_input":"2023-07-12T15:57:53.802742Z","iopub.status.idle":"2023-07-12T15:58:02.488888Z","shell.execute_reply.started":"2023-07-12T15:57:53.802713Z","shell.execute_reply":"2023-07-12T15:58:02.487765Z"},"trusted":true},"execution_count":null,"outputs":[]}]}