{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# <span style=\"color:#674AB3;\"> Introduction </span> 🙂\n\n> #### <span style=\"color:darkblue;\"> \"With great power comes great responsibility!\" </span>\n\n<span style=\"font-family: 'Lucida Console';\"> There is an inherent power in <a href=\"https://news.stanford.edu/2019/08/22/the-power-of-language-how-words-shape-people-culture/\">language that contribute immensely towards shaping people's decisions whether it be the individual perception that may be biased based on beliefs, resulting in the actions taken in human societies</a>. The famous last words of Uncle Ben in the Spiderman movie should be applied at their meaningful best when it comes to handling the power of Machine Learning and Data Science expertise unraveled from the brilliant minds who participate in designing solutions during Kaggle competitions. A few weeks after the Hiroshima bombing using the nuclear invention of his own moonshot project to build an atomic bomb in the top-secret lab, <a href=\"https://www.nationalgeographic.com/history/article/julius-robert-oppenheimer-atomic-bomb-legacy\"> American scientist J. Robert Oppenheimer wrote a letter to the Secretary of War warning that “the safety of this nation…cannot lie wholly or even primarily in its scientific or technical prowess. It can be based only on making future wars impossible.”</a>\nThe 2023 Kaggle AI report competition does promise quite an analytical insight into the recent progress and trends of machine learning enthusiasts. The progress evaluation done in the competition notebooks are sure to contribute towards AI in an impactful way, and hence the ideology by Isaac Asimov to ensure that the <a href=\"https://www.britannica.com/topic/I-Robot\"> artificially intelligent machines being programmed with ethics in his science fiction way back in the 1900s</a> needs great emphasis in today's world to handle the innovations responsibly. </span>\n\n<span style=\"font-family: 'Lucida Console';\"><a href=\"https://www.makeuseof.com/beginners-guide-to-kaggle/\">Kaggle</a>, the well-known host of highly skill-demanding and rewarding machine learning and data science competitions, is just the right playground for innovation and skill-share pushing the boundaries of what is possible. The responsibility that comes with such highly rewarding opportunities in contributing towards intelligent technologies, an all-in-one place at Kaggle, would propel us to develop high-performance models embracing ethical practices using the improvised library versions. The development of cutting-edge innovative solutions during the Kaggle competitions hosted in a professional environment is a promise that amplifies the spirit of the thrill of discovery during the competition timeline. </span>\n\n***","metadata":{}},{"cell_type":"markdown","source":"# <span style=\"color:#674AB3;\"> Delving into Meta Kaggle Dataset </span> 👀\n\n*** \n\n<span style=\"font-family: 'Lucida Console';\"> <a href=\"https://www.kaggle.com/docs/competitions\">Competitions at Kaggle</a> not only drive excellence to encourage innovation and thereby promote technological advancements in the industry but also inspires the participants in goal-setting and striving for success on a personal level. It is a great way to learn, share and collaborate resulting in personal skill development to meet the high technical standards essential to tackle real-world challenges. The recognition of winning can foster career progress along with a guaranteed professional reputation, high financial rewards, and credibility. In this AI report, I would like to present machine learning enthusiasts with the comprehensive analytical conclusions that can be formulated from the <a href=\"https://www.kaggle.com/datasets/kaggle/meta-kaggle\">Meta Kaggle - Kaggle's public data on competitions, users, submission scores, and kernels</a> in relation to the competitions held in the timeline of past two years on the Kaggle platform. I have also included a conclusion at the end for the non-experts highlighting a short summary of the core insights as observed from the Meta Kaggle dataset to embark on a successful journey from Kaggle Novice to being a Kaggle Grandmaster in the prestigious platform. Some short reads with references to understand the relevance and impact of AI in healthcare, finances, NLP, and Computer Vision, which were encouraged in Kaggle competitions in recent years, have been included along with it to explore the positive influence of AI in our day-to-day living.</span>\n\n<span style=\"font-family: 'Lucida Console';\"> The kernels published using the Meta Kaggle dataset is having an expected update frequency on a daily basis and can help the machine learning enthusiasts and data science community to strategize their way to become a Competitions Grandmaster in the Kaggle community of developers. The analysis of this dataset, not only provides a lens-view detailed insight into the best skills in a winning team but also can act as a winner's guide to choosing the best evaluation metrics for the next data science project. </span>\n\n***","metadata":{}},{"cell_type":"markdown","source":"### <span style=\"color:#674AB3;\"> Let's import some useful libraries! </span> 🕸\n\n***\n<span style=\"font-family: 'Lucida Console';\"> Machine learning libraries are open-source solutions that help us to tackle complex problems and extract insights from vast amounts of data. These libraries can help us to draw useful insights from the Meta-Kaggle dataset that provide a detailed glimpse into the world of Competitions hosted by Kaggle during the past two years. </span>\n***","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport matplotlib.pyplot as plt\nimport datetime\nimport seaborn as sns","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2023-07-19T09:26:16.431231Z","iopub.execute_input":"2023-07-19T09:26:16.432451Z","iopub.status.idle":"2023-07-19T09:26:17.242460Z","shell.execute_reply.started":"2023-07-19T09:26:16.432377Z","shell.execute_reply":"2023-07-19T09:26:17.241155Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# <span style=\"color:#674AB3;\"> 1. Exploring Kaggle Competitions... </span> 🏆\n\n***\n\n### <span style=\"color: #0020FF;\"> 1.1 Analyzing Meta Kaggle CompetitionTags </span>\n\n<span style=\"font-family: 'Lucida Console';\"> CompetitionTags dataset from Meta Kaggle provides the details like Id, Competition Id, and Tag Id that refers to identifiers that help uniquely identify and organize various entities within the competition. The Id refers to the individual entries submitted to the Kaggle competition with the Competition Id mentioned in the dataset. Tags help in organizing and categorizing competitions, making it easier for participants and users to discover and explore competitions of their interest. The Tag Id serves as a reference to a specific tag associated with a competition. These IDs play a crucial role in maintaining the integrity and organization of data within Kaggle competitions. This assists in efficient retrieval, linking, and analysis of different entries to the competitions. </span>\n\n***","metadata":{}},{"cell_type":"code","source":"# Load the CompetitionTags Meta Kaggle dataset\nCompetitionTags = pd.read_csv('/kaggle/input/meta-kaggle/CompetitionTags.csv',low_memory=False)\n\n# Display the CompetitionTags Meta Kaggle dataset\nn_rows = CompetitionTags.shape[0]\nCompetitionTags.head(n_rows).T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:02:46.732613Z","iopub.execute_input":"2023-07-19T09:02:46.733083Z","iopub.status.idle":"2023-07-19T09:02:46.818583Z","shell.execute_reply.started":"2023-07-19T09:02:46.733049Z","shell.execute_reply":"2023-07-19T09:02:46.817272Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"***\n<span style=\"font-family: 'Lucida Console';\"> The heatmap provides a color-coded representation of the data, making it easy to identify patterns and relationships. From the analysis, it can be observed that Tag Id 14101 shows a cluster of strong positive correlation amongst almost all the Competition Ids in the Meta Kaggle CompetitionTags.csv data. The Tag Id 14101 corresponds to tabular data used in the various Competitions held on the Kaggle platform.</span>\n\n**A Note..**\n\n* [How to search Kaggle Competitions using Tag Id](https://www.kaggle.com/discussions/product-feedback/301940#1657528) \n\n***","metadata":{}},{"cell_type":"code","source":"# Create a cross-tabulation or pivot table to calculate the frequency or count\ncross_tab = pd.crosstab(CompetitionTags['TagId'], CompetitionTags['CompetitionId'])\n\n# Plot the heatmap using seaborn\nplt.figure(figsize=(10, 8))\nsns.heatmap(cross_tab, cmap='YlGnBu')\nplt.title('Heatmap of Tag Id and Competition Id')\nplt.xlabel('Competition Id')\nplt.ylabel('Tag Id')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:02:54.524232Z","iopub.execute_input":"2023-07-19T09:02:54.524678Z","iopub.status.idle":"2023-07-19T09:02:55.805388Z","shell.execute_reply.started":"2023-07-19T09:02:54.524645Z","shell.execute_reply":"2023-07-19T09:02:55.804354Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### <span style=\"color: #0020FF;\"> 1.2 Analyzing Meta Kaggle Competitions </span>\n\n***\n\n<span style=\"font-family: 'Lucida Console';\"> Competitions dataset from Meta Kaggle provides the details like 'EnabledDate', 'EvaluationAlgorithmAbbreviation', 'EvaluationAlgorithmName', 'EvaluationAlgorithmDescription', 'RewardType' etc. of the competitions launched by Kaggle. Let's delve deeper into further analysis of the dataset. </span>\n***","metadata":{}},{"cell_type":"code","source":"# Load the Competition Meta Kaggle dataset\nCompetitions = pd.read_csv('/kaggle/input/meta-kaggle/Competitions.csv',low_memory=False)\n\n# Filetered data for the recent two years\nCompetitions['EnabledDate'] = pd.to_datetime(Competitions['EnabledDate'])  # Convert to datetime \nComp_filtered_df = Competitions[(Competitions['EnabledDate'] >= '2021-01-01') & (Competitions['EnabledDate'] <= '2023-12-31')]\n\nn_rows = Comp_filtered_df.shape[0]\nprint(\"The total number of Competitions during the recent two years '21-'23 : \",n_rows)","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:33:20.590590Z","iopub.execute_input":"2023-07-19T09:33:20.591141Z","iopub.status.idle":"2023-07-19T09:33:21.424302Z","shell.execute_reply.started":"2023-07-19T09:33:20.591101Z","shell.execute_reply":"2023-07-19T09:33:21.423031Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Comp_filtered_df.head(n_rows).T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:03:06.414049Z","iopub.execute_input":"2023-07-19T09:03:06.415194Z","iopub.status.idle":"2023-07-19T09:03:06.494598Z","shell.execute_reply.started":"2023-07-19T09:03:06.415144Z","shell.execute_reply":"2023-07-19T09:03:06.493038Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"***\n<span style=\"font-family: 'Lucida Console';\"> Let's look into the algorithms used for the evaluation of the models trained for winning the competitions according to Meta Kaggle for the past two years. </span>\n***","metadata":{}},{"cell_type":"code","source":"# Grouping the data by the Evaluation Algorithm details involved in the Competitions dataset\nComp_grouped_df = Comp_filtered_df.groupby(['EvaluationAlgorithmAbbreviation','EvaluationAlgorithmName', 'EvaluationAlgorithmDescription']).size().reset_index(name='Count')\n\n# Sorting in the descending order based on the frequency of implementations\nComp_sorted_df = Comp_grouped_df.sort_values(by='Count', ascending=False)\nn_rows = Comp_sorted_df.shape[0]\nComp_sorted_df.head(n_rows).T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:33:31.942400Z","iopub.execute_input":"2023-07-19T09:33:31.942944Z","iopub.status.idle":"2023-07-19T09:33:31.984210Z","shell.execute_reply.started":"2023-07-19T09:33:31.942893Z","shell.execute_reply":"2023-07-19T09:33:31.982744Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"***\n<span style=\"font-family: 'Lucida Console';\"> The different reward types offered by the Competitions over the years from the beginning of the Kaggle competitions have been compared with the recent trends over the past two years and have been analyzed here using the single variable analytical bar plot visualization. The lucrative rewards to support budding talents to establish their success in machine learning and data science is a motivation in themselves.</span>\n\n**An Update**\n\n* [Kaggle Jobs Board update](https://www.kaggle.com/jobs-board-closed)\n\n***","metadata":{}},{"cell_type":"code","source":"# Visualize competition rewards over the years\nreward_counts = Competitions['RewardType'].value_counts()\nplt.figure(figsize=(5,3))\nreward_counts.plot(kind='bar', color='blue')\nplt.xlabel('Reward Type')\nplt.ylabel('Count')\nplt.title('Competition Reward Types')\nplt.show()\n\n# Visualize competition rewards over the recent two years\nreward_counts = Comp_filtered_df['RewardType'].value_counts()\nplt.figure(figsize=(5, 3))\nreward_counts.plot(kind='bar', color='blue')\nplt.xlabel('Reward Type')\nplt.ylabel('Count')\nplt.title('Competition Reward Types')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:03:24.108106Z","iopub.execute_input":"2023-07-19T09:03:24.108563Z","iopub.status.idle":"2023-07-19T09:03:24.773258Z","shell.execute_reply.started":"2023-07-19T09:03:24.108529Z","shell.execute_reply":"2023-07-19T09:03:24.772042Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# <span style=\"color: #674AB3;\"> 2. Exploring Meta Kaggle Datasets </span> 📁\n\n<span style=\"font-family: 'Lucida Console';\"> Kaggle supports sharing of quality data in a variety of dataset publication formats. Here are a few useful resources that can be very helpful. </span>\n    \n\n**More on Datasets in Kaggle**\n    \n1. [About Kaggle Datasets](https://www.kaggle.com/docs/datasets)\n1. [How to use dataset in Kaggle competitions](https://www.kaggle.com/docs/competitions-setup#how-kaggle-competitions-work)\n1. [How to share research data on Kaggle](https://youtu.be/wmbDkhPp5qs)\n1. [How to Update a Dataset On Kaggle | Kaggle](https://youtu.be/m--1LEvNmKw)\n1. [How to Upload a Dataset to Kaggle | Kaggle](https://youtu.be/YryPL0GnfTc)\n","metadata":{}},{"cell_type":"markdown","source":"***\n### <span style=\"color: #0020FF;\"> 2.1 Analyzing Meta Kaggle DatasetTags </span>\n\n<span style=\"font-family: 'Lucida Console';\"> DatasetTags from Meta Kaggle provides an insight into the datasets used in the various competitions using 'Tag Id', 'Dataset Id' etc. </span>\n\n***","metadata":{}},{"cell_type":"code","source":"# Load the CompetitionTags Meta Kaggle dataset\nDatasetTags = pd.read_csv('/kaggle/input/meta-kaggle/DatasetTags.csv',low_memory=False)\n\n# Display the CompetitionTags Meta Kaggle dataset\nn_rows = DatasetTags.shape[0]\nDatasetTags.head(n_rows).T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:03:33.866705Z","iopub.execute_input":"2023-07-19T09:03:33.867166Z","iopub.status.idle":"2023-07-19T09:03:34.111616Z","shell.execute_reply.started":"2023-07-19T09:03:33.867132Z","shell.execute_reply":"2023-07-19T09:03:34.110525Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"***\n<span style=\"font-family: 'Lucida Console';\"> Let's calculate the pairwise correlation between columns of the DataFrame. The resulting correlation matrix shows the correlation coefficients between Id, TagId, and DatasetId in the dataset. </span>\n***","metadata":{}},{"cell_type":"code","source":"correlation_matrix = DatasetTags.corr()\n\n# Display the correlation matrix\nprint(correlation_matrix)\n","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:03:36.518248Z","iopub.execute_input":"2023-07-19T09:03:36.518613Z","iopub.status.idle":"2023-07-19T09:03:36.547475Z","shell.execute_reply.started":"2023-07-19T09:03:36.518584Z","shell.execute_reply":"2023-07-19T09:03:36.546134Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### <span style=\"color: #0020FF;\"> 2.2 Analyzing Meta Kaggle DatasetTaskSubmissions </span>\n\n***\n\n<span style=\"font-family: 'Lucida Console';\"> DatasetTaskSubmissions dataset from Meta Kaggle provides information like 'CreationDate', 'DatasetId'etc. regarding the dataset tasks submitted to Kaggle. </span>\n\n***\n","metadata":{}},{"cell_type":"code","source":"DatasetTaskSubmissions = pd.read_csv('/kaggle/input/meta-kaggle/DatasetTaskSubmissions.csv',low_memory=False)\nn_rows = DatasetTaskSubmissions.shape[0]\nDatasetTaskSubmissions.head(n_rows).T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:03:46.348167Z","iopub.execute_input":"2023-07-19T09:03:46.348614Z","iopub.status.idle":"2023-07-19T09:03:46.472085Z","shell.execute_reply.started":"2023-07-19T09:03:46.348582Z","shell.execute_reply":"2023-07-19T09:03:46.470810Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Filetered data for the recent two years\nDatasetTaskSubmissions['CreationDate'] = pd.to_datetime(DatasetTaskSubmissions['CreationDate'])  # Convert to datetime \nDat_filtered_df = DatasetTaskSubmissions[(DatasetTaskSubmissions['CreationDate'] >= '2021-01-01') & (DatasetTaskSubmissions['CreationDate'] <= '2023-12-31')]\n\nprint(\"The total number of dataset task submissions : \",DatasetTaskSubmissions['CreationDate'].shape[0])\nn_rows = Dat_filtered_df.shape[0]\nprint(\"The total number of dataset tasks submitted during the recent two years '21-'23 : \",n_rows)\n\n# The details of the datasets submitted recently in two years\nDat_filtered_df.head(n_rows).T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:03:50.529155Z","iopub.execute_input":"2023-07-19T09:03:50.529642Z","iopub.status.idle":"2023-07-19T09:03:51.900073Z","shell.execute_reply.started":"2023-07-19T09:03:50.529603Z","shell.execute_reply":"2023-07-19T09:03:51.898879Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### <span style=\"color: #0020FF;\"> 2.3 Analyzing Meta Kaggle DatasetTasks </span>\n\n***\n\n<span style=\"font-family: 'Lucida Console';\"> DatasetTasks dataset from Meta Kaggle gives an overview of the description of the dataset according to the timeline it was added to the Kaggle environment. </span>\n\n***\n","metadata":{}},{"cell_type":"code","source":"DatasetTasks = pd.read_csv('/kaggle/input/meta-kaggle/DatasetTasks.csv',low_memory = False)\nDatasetTasks.head(DatasetTasks.shape[0]).T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:03:59.845003Z","iopub.execute_input":"2023-07-19T09:03:59.846268Z","iopub.status.idle":"2023-07-19T09:04:00.133810Z","shell.execute_reply.started":"2023-07-19T09:03:59.846201Z","shell.execute_reply":"2023-07-19T09:04:00.132456Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Filetered data for the recent two years\nDatasetTasks['CreationDate'] = pd.to_datetime(DatasetTasks['CreationDate'])  # Convert to datetime \nDat_filtered_df = DatasetTasks[(DatasetTasks['CreationDate'] >= '2021-01-01') & (DatasetTasks['CreationDate'] <= '2023-12-31')]\n\nprint(\"The total number of dataset tasks : \",DatasetTasks['CreationDate'].shape[0])\nn_rows = Dat_filtered_df.shape[0]\nprint(\"The total number of dataset tasks during the recent two years '21-'23 : \",n_rows)\n\n# Grouping the data by the Evaluation Algorithm details involved in the Competitions dataset\nDat_grouped_df = Dat_filtered_df.groupby(['CreationDate','DatasetId','Title','Subtitle', 'Description']).size().reset_index(name='Count')\n\n# Sorting in the descending order based on the frequency of implementations\nDat_sorted_df = Dat_grouped_df.sort_values(by='Count', ascending=False)\nn_rows = Dat_sorted_df.shape[0]\n\n# The details of the datasets submitted recently in two years\nDat_sorted_df.head(n_rows).T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:04:13.305522Z","iopub.execute_input":"2023-07-19T09:04:13.306016Z","iopub.status.idle":"2023-07-19T09:04:14.155201Z","shell.execute_reply.started":"2023-07-19T09:04:13.305972Z","shell.execute_reply":"2023-07-19T09:04:14.153830Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### <span style=\"color: #0020FF;\"> 2.4 Analyzing Meta Kaggle Dataset Versions </span>\n\n***\n\n<span style=\"font-family: 'Lucida Console';\"> DatasetVersions from Meta Kaggle provides the details like 'DatasetId', 'Title', 'Subtitle', 'Description', 'Slug' etc. which helps to get to know about the overview of the competition hosted in the Kaggle. </span>\n\n***","metadata":{}},{"cell_type":"code","source":"DatasetVersions = pd.read_csv('/kaggle/input/meta-kaggle/DatasetVersions.csv', low_memory = False)\nDatasetVersions.head(DatasetVersions.shape[0]).T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:04:22.195712Z","iopub.execute_input":"2023-07-19T09:04:22.196213Z","iopub.status.idle":"2023-07-19T09:04:51.169535Z","shell.execute_reply.started":"2023-07-19T09:04:22.196178Z","shell.execute_reply":"2023-07-19T09:04:51.168347Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Filetered data for the recent two years\nDatasetVersions['CreationDate'] = pd.to_datetime(DatasetVersions['CreationDate'])  # Convert to datetime \nDat_filtered_df = DatasetVersions[(DatasetVersions['CreationDate'] >= '2021-01-01') & (DatasetVersions['CreationDate'] <= '2023-12-31')]\n\nprint(\"The total number of data : \",DatasetVersions['CreationDate'].shape[0])\nn_rows = Dat_filtered_df.shape[0]\nprint(\"The total number of data during the recent two years '21-'23 : \",n_rows)\n\n# Grouping the data by the Evaluation Algorithm details involved in the Competitions dataset\nDat_grouped_df = Dat_filtered_df.groupby(['CreationDate','DatasetId','Title','Subtitle', 'Description','VersionNotes']).size().reset_index(name='Count')\n\n# Sorting in the descending order based on the frequency of implementations\nDat_sorted_df = Dat_grouped_df.sort_values(by='Count', ascending=False)\nn_rows = Dat_sorted_df.shape[0]\n\n# The details of the datasets submitted recently in two years\nDat_sorted_df.head(n_rows).T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:06:46.600031Z","iopub.execute_input":"2023-07-19T09:06:46.600407Z","iopub.status.idle":"2023-07-19T09:06:49.949751Z","shell.execute_reply.started":"2023-07-19T09:06:46.600374Z","shell.execute_reply":"2023-07-19T09:06:49.948563Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### <span style=\"color: #0020FF;\"> 2.5 Analyzing Meta Kaggle DatasetVotes </span>\n\n***\n\n<span style=\"font-family: 'Lucida Console';\"> DatasetVotes dataset from Meta Kaggle provides the details like VoteDate, Id, etc. which assists in identifying the most popular and reliable datasets preferred by different Kaggle users. </span>\n\n***","metadata":{}},{"cell_type":"code","source":"DatasetVotes = pd.read_csv('/kaggle/input/meta-kaggle/DatasetVotes.csv', low_memory=False)\n# Filtered data for the recent two years\nDatasetVotes['VoteDate'] = pd.to_datetime(DatasetVotes['VoteDate'])  # Convert to datetime \nDat_filtered_df = DatasetVotes[(DatasetVotes['VoteDate'] >= '2021-01-01') & (DatasetVotes['VoteDate'] <= '2023-12-31')]\n\nprint(\"The total number of data : \",DatasetVotes['VoteDate'].shape[0])\nn_rows = Dat_filtered_df.shape[0]\nprint(\"The total number of data during the recent two years '21-'23 : \",n_rows)\nDat_filtered_df.head(n_rows).T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:06:49.951469Z","iopub.execute_input":"2023-07-19T09:06:49.951959Z","iopub.status.idle":"2023-07-19T09:07:00.224230Z","shell.execute_reply.started":"2023-07-19T09:06:49.951895Z","shell.execute_reply":"2023-07-19T09:07:00.222995Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### <span style=\"color: #0020FF;\"> 2.6 Analyzing Meta Kaggle Datasets </span>\n\n***\n\n<span style=\"font-family: 'Lucida Console';\"> 'Datasets' dataset from Meta Kaggle provides the details like Id, CreationDate, ForumId, OwnerUserId, etc. that give an immediate reference to the details of the uploaded dataset over the '21-'23 timeline, all-in-one-place! </span>\n\n***\n","metadata":{}},{"cell_type":"code","source":"Datasets = pd.read_csv('/kaggle/input/meta-kaggle/Datasets.csv', low_memory=False)\n# Filtered data for the recent two years\nDatasets['CreationDate'] = pd.to_datetime(Datasets['CreationDate'])  # Convert to datetime \nDat_filtered_df = Datasets[(Datasets['CreationDate'] >= '2021-01-01') & (Datasets['CreationDate'] <= '2023-12-31')]\n\nprint(\"The total number of data : \",Datasets['CreationDate'].shape[0])\nn_rows = Dat_filtered_df.shape[0]\nprint(\"The total number of data during the recent two years '21-'23 : \",n_rows)\nDat_filtered_df.head(n_rows).T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:07:00.225834Z","iopub.execute_input":"2023-07-19T09:07:00.226277Z","iopub.status.idle":"2023-07-19T09:07:32.541957Z","shell.execute_reply.started":"2023-07-19T09:07:00.226244Z","shell.execute_reply":"2023-07-19T09:07:32.540720Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### <span style=\"color: #0020FF;\"> 2.7 Analyzing Meta Kaggle DataSources </span>\n\n***\n\n<span style=\"font-family: 'Lucida Console';\"> DataSources dataset from Meta Kaggle provides the details like Id, CreationDate, VersionId, etc. giving a more organized way to sort the sources of the datasets shared on the Kaggle platform. </span>\n\n***","metadata":{}},{"cell_type":"code","source":"Datasources = pd.read_csv('/kaggle/input/meta-kaggle/Datasources.csv', low_memory=False)\n# Filtered data for the recent two years\nDatasources['CreationDate'] = pd.to_datetime(Datasources['CreationDate'])  # Convert to datetime \nDat_filtered_df = Datasources[(Datasources['CreationDate'] >= '2021-01-01') & (Datasources['CreationDate'] <= '2023-12-31')]\n\nprint(\"The total number of data : \",Datasets['CreationDate'].shape[0])\nn_rows = Dat_filtered_df.shape[0]\nprint(\"The total number of data during the recent two years '21-'23 : \",n_rows)\nDat_filtered_df.head(n_rows).T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:07:32.543451Z","iopub.execute_input":"2023-07-19T09:07:32.543809Z","iopub.status.idle":"2023-07-19T09:08:04.675818Z","shell.execute_reply.started":"2023-07-19T09:07:32.543772Z","shell.execute_reply":"2023-07-19T09:08:04.674657Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#  <span style=\"color: #674AB3;\"> 3. Exploring Meta Kaggle Episodes </span> 🧩\n\n<span style=\"font-family: 'Lucida Console';\"> The <a href=\"https://github.com/Kaggle/kaggle-environments\">Kaggle environment</a> was created to evaluate episodes. Episodes typically have a start date and an end date, and they are often divided into several rounds or phases. A Kaggle competition can typically consist of individual stages called episodes intended to help participants iteratively improve their machine-learning models for the dataset involved in the challenges according to the specified time rubric. </span>\n\n**Some specific episodes in recent years...**\n\n1. [2023 edition of Kaggle's Playground Series!](https://www.kaggle.com/discussions/general/375342) \n2. [Playground series in Competitions](https://www.kaggle.com/competitions?hostSegmentIdFilter=8)\n","metadata":{}},{"cell_type":"markdown","source":"***\n### <span style=\"color: #0020FF;\"> Analyzing Meta Kaggle Episodes and EpisodeAgents </span>\n\n<span style=\"font-family: 'Lucida Console';\"> Episodes dataset from Meta Kaggle provides the details like 'Id', 'CreateTime', 'CompetitionId' etc. The interactive dashboard available for the <a href=\"https://www.kaggle.com/datasets/kaggle/meta-kaggle?select=Episodes.csv&sort=recent-comments\">Meta Kaggle Episodes dataset </a> provides overall statistical analytics of the data. </span>\n\n\n<img src=\"https://drive.google.com/uc?id=1H8xk0iidw9kbzuu-Ws7uodnALf-EpNDs\" alt=\"drawing\" width=\"700\" height = \"450\"/>\n\n<span style=\"font-family: 'Lucida Console';\"> <a href=\"https://www.kaggle.com/datasets/kaggle/meta-kaggle?select=EpisodeAgents.csv&sort=recent-comments\">EpisodeAgents dataset from Meta Kaggle</a> provides information about 'EpisodeId', 'InitialScore' etc.</span>\n\n**Additional reference**\n\n* [Analysis of every Playground Series Episode](#https://www.kaggle.com/code/mattop/analysis-of-every-playground-series-episode)\n***\n","metadata":{}},{"cell_type":"markdown","source":"# <span style=\"color: #674AB3;\"> 4. Exploring Meta Kaggle Forums </span> 📜\n\n<span style=\"font-family: 'Lucida Console';\"> Competitions forums of Kaggle pave the way to gain clarity by raising queries related to the competitions hosted by Kaggle. Discussion Forums open the way to explore Kaggle resources in a unique way, in addition to the opportunity given to be a part of the community by sharing ideas, feedback, and a lot more. </span>\n\n**More on Kaggle Forums**\n\n1. [Kaggle Competition Forums](https://www.kaggle.com/docs/competitions) \n1. <span style=\"color: blue;\">Kaggle Discussion Forums </span>\n    * [General](https://www.kaggle.com/discussions/general)\n    * [Getting Started](https://www.kaggle.com/discussions/getting-started)\n    * [Product Feedback](https://www.kaggle.com/discussions/product-feedback)\n    * [Questions & Answers](https://www.kaggle.com/discussions/questions-and-answers)\n    * [Competition Hosting](https://www.kaggle.com/discussions/competition-hosting)\n1. [Predict Kaggle forum message medals](https://www.kaggle.com/code/danofer/predict-kaggle-forum-message-medals)\n1. [A look at sentiment on the Kaggle Forums](https://www.kaggle.com/code/jtrotman/a-look-at-sentiment-on-the-kaggle-forums).\n    \n","metadata":{}},{"cell_type":"markdown","source":"***\n### <span style=\"color: #0020FF;\"> 4.1 Analyzing Meta Kaggle ForumMessagesVotes </span>\n\n<span style=\"font-family: 'Lucida Console';\"> ForumMessagesVotes dataset from Meta Kaggle provides the details like 'ForumMessageId', 'VoteDate' etc. Let's look at the dataset updated on Meta Kaggle for the recent two years timeline. </span>\n\n***","metadata":{}},{"cell_type":"code","source":"ForumMessageVotes = pd.read_csv('/kaggle/input/meta-kaggle/ForumMessageVotes.csv', low_memory=False)\n# Filtered data for the recent two years\nForumMessageVotes['VoteDate'] = pd.to_datetime(ForumMessageVotes['VoteDate'])  # Convert to datetime \nForum_filtered_df = ForumMessageVotes[(ForumMessageVotes['VoteDate'] >= '2021-01-01') & (ForumMessageVotes['VoteDate'] <= '2023-12-31')]\n\nprint(\"The total number of data : \",ForumMessageVotes['VoteDate'].shape[0])\nn_rows = Forum_filtered_df.shape[0]\nprint(\"The total number of data during the recent two years '21-'23 : \",n_rows)\nForum_filtered_df.head(n_rows).T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:08:19.691244Z","iopub.execute_input":"2023-07-19T09:08:19.691602Z","iopub.status.idle":"2023-07-19T09:08:32.740796Z","shell.execute_reply.started":"2023-07-19T09:08:19.691572Z","shell.execute_reply":"2023-07-19T09:08:32.739420Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"***\n### <span style=\"color: #0020FF;\"> 4.2 Analyzing Meta Kaggle ForumMessages </span>\n\n<span style=\"font-family: 'Lucida Console';\"> The ForumMessages dataset from Meta Kaggle provides the details like 'ForumMessageId', 'VoteDate' etc. which can give an insight into the forum responses inquired over the recent two-year timeline using the analysis done below.</span>\n\n***\n","metadata":{}},{"cell_type":"code","source":"ForumMessages = pd.read_csv('/kaggle/input/meta-kaggle/ForumMessages.csv', low_memory=False)\n\n# Filtered data for the recent two years\nForumMessages['PostDate'] = pd.to_datetime(ForumMessages['PostDate'])  # Convert to datetime \nForum_filtered_df = ForumMessages[(ForumMessages['PostDate'] >= '2021-01-01') & (ForumMessages['PostDate'] <= '2023-12-31')]\n\nprint(\"The total number of data : \",ForumMessages['PostDate'].shape[0])\nn_rows = Forum_filtered_df.shape[0]\nprint(\"The total number of data during the recent two years '21-'23 : \",n_rows)\nForum_filtered_df.head(n_rows).T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:08:43.252041Z","iopub.execute_input":"2023-07-19T09:08:43.252634Z","iopub.status.idle":"2023-07-19T09:13:24.534816Z","shell.execute_reply.started":"2023-07-19T09:08:43.252590Z","shell.execute_reply":"2023-07-19T09:13:24.533633Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"***\n### <span style=\"color: #0020FF;\"> 4.3 Analyzing Meta Kaggle ForumTopics </span>\n\n<span style=\"font-family: 'Lucida Console';\"> ForumTopics dataset from Meta Kaggle provides the details like 'ForumId', 'CreationDate' etc. can be helpful in gaining a statistical insight as to how useful the forum discussions have been to the Kagglers in the recent two-year timespan. </span>\n\n***\n","metadata":{}},{"cell_type":"code","source":"ForumTopics = pd.read_csv('/kaggle/input/meta-kaggle/ForumTopics.csv', low_memory=False)\n\n# Filtered data for the recent two years\nForumTopics['CreationDate'] = pd.to_datetime(ForumTopics['CreationDate'])  # Convert to datetime \nForum_filtered_df = ForumTopics[(ForumTopics['CreationDate'] >= '2021-01-01') & (ForumTopics['CreationDate'] <= '2023-12-31')]\n\nprint(\"The total number of data : \",ForumTopics['CreationDate'].shape[0])\nn_rows = Forum_filtered_df.shape[0]\nprint(\"The total number of data during the recent two years '21-'23 : \",n_rows)\nForum_filtered_df.head(n_rows).T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:14:10.084788Z","iopub.execute_input":"2023-07-19T09:14:10.085350Z","iopub.status.idle":"2023-07-19T09:14:55.064992Z","shell.execute_reply.started":"2023-07-19T09:14:10.085299Z","shell.execute_reply":"2023-07-19T09:14:55.063524Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"***\n### <span style=\"color: #0020FF;\"> 4.4 Analyzing Meta Kaggle Forums </span>\n\n<span style=\"font-family: 'Lucida Console';\"> Forums dataset from Meta Kaggle provides the details like 'Id', 'ParentForumId' and 'Title' provides us with a reliable data source to identify the Id associated with each competition title in the Kaggle. </span>\n\n\n***\n","metadata":{}},{"cell_type":"code","source":"Forums = pd.read_csv('/kaggle/input/meta-kaggle/Forums.csv', low_memory=False)\nForums.head(Forums.shape[0]).T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:14:56.965636Z","iopub.execute_input":"2023-07-19T09:14:56.966856Z","iopub.status.idle":"2023-07-19T09:14:58.716421Z","shell.execute_reply.started":"2023-07-19T09:14:56.966802Z","shell.execute_reply":"2023-07-19T09:14:58.715267Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# <span style=\"color: #674AB3;\"> 5. Exploring Meta Kaggle Kernels </span> 📊\n\n<span style=\"font-family: 'Lucida Console';\"> Kernels are an integral part of the Kaggle platform, allowing participants to share code, insights, and techniques. The Kaggle kernels IDE for Data Scientists is used for creating <a href=\"https://www.kaggle.com/code\">notebooks</a> for submission into the Kaggle competitions. </span>\n\n<span style=\"font-family: 'Lucida Console';\"> The Kaggle API can be used to create new Notebooks and Notebook versions on Kaggle using a command line. </span>\n\n**More reading on Kaggle Notebooks**\n\n1. [Kaggle Notebooks](\"https://www.kaggle.com/docs/notebooks\")\n1. [Kaggle API](https://www.kaggle.com/docs/api)\n","metadata":{}},{"cell_type":"markdown","source":"***\n\n### <span style=\"color: #0020FF;\"> 5.1 Analyzing Meta Kaggle KernelLanguages </span>\n\n<span style=\"font-family: 'Lucida Console';\"> KernelLanguages dataset from Meta Kaggle shows typically notebooks created using Python and R languages. </span>\n\n***","metadata":{}},{"cell_type":"code","source":"KernelLanguages = pd.read_csv('/kaggle/input/meta-kaggle/KernelLanguages.csv', low_memory=False)\nKernelLanguages.head(KernelLanguages.shape[0]).T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:14:59.473847Z","iopub.execute_input":"2023-07-19T09:14:59.474237Z","iopub.status.idle":"2023-07-19T09:14:59.505660Z","shell.execute_reply.started":"2023-07-19T09:14:59.474205Z","shell.execute_reply":"2023-07-19T09:14:59.504418Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### <span style=\"color: #0020FF;\"> 5.2 Analyzing Meta Kaggle KernelTags </span>\n\n***\n\n<span style=\"font-family: 'Lucida Console';\"> KernelTags dataset from Meta Kaggle provides the details like 'Id', 'KernelId', and 'TagId'. There are 469 unique TagId present in the dataset along with 255229 unique KernelId.</span>\n\n***","metadata":{}},{"cell_type":"code","source":"KernelTags = pd.read_csv('/kaggle/input/meta-kaggle/KernelTags.csv', low_memory=False)\nKernelTags.head().T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:14:58.717835Z","iopub.execute_input":"2023-07-19T09:14:58.718813Z","iopub.status.idle":"2023-07-19T09:14:59.369423Z","shell.execute_reply.started":"2023-07-19T09:14:58.718774Z","shell.execute_reply":"2023-07-19T09:14:59.368033Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"KernelTags.nunique()","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:14:59.371393Z","iopub.execute_input":"2023-07-19T09:14:59.371918Z","iopub.status.idle":"2023-07-19T09:14:59.472380Z","shell.execute_reply.started":"2023-07-19T09:14:59.371869Z","shell.execute_reply":"2023-07-19T09:14:59.471114Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### <span style=\"color: #0020FF;\"> 5.3 Analyzing Meta Kaggle KernelVersionCompetitionSources </span>\n\n***\n\n<span style=\"font-family: 'Lucida Console';\"> KernelVersionCompetitionSources dataset from Meta Kaggle provides the details like 'Id', 'KernelVersionId', and 'SourceCompetitionId'. There are 1696 unique SourceCompetitionId present in the dataset along with 2105943 unique KernelVersionId as observed from the analysis. </span>\n\n\n***","metadata":{}},{"cell_type":"code","source":"KernelVersionCompetitionSources = pd.read_csv('/kaggle/input/meta-kaggle/KernelVersionCompetitionSources.csv',low_memory = False)\nKernelVersionCompetitionSources.head().T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:14:59.507382Z","iopub.execute_input":"2023-07-19T09:14:59.507892Z","iopub.status.idle":"2023-07-19T09:15:01.409443Z","shell.execute_reply.started":"2023-07-19T09:14:59.507847Z","shell.execute_reply":"2023-07-19T09:15:01.407879Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"KernelVersionCompetitionSources.nunique()","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:15:01.416079Z","iopub.execute_input":"2023-07-19T09:15:01.416560Z","iopub.status.idle":"2023-07-19T09:15:02.242703Z","shell.execute_reply.started":"2023-07-19T09:15:01.416526Z","shell.execute_reply":"2023-07-19T09:15:02.241247Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### <span style=\"color: #0020FF;\"> 5.4 Analyzing Meta Kaggle KernelVersionDatasetSources </span>\n\n***\n\n<span style=\"font-family: 'Lucida Console';\"> KernelVersionDatasetSources dataset from Meta Kaggle provides the details like 'Id', 'KernelVersionId', and 'SourceCompetitionId'. There are 267844 unique SourceCompetitionId present in the dataset along with 4296811 unique KernelVersionId.</span>\n\n***","metadata":{}},{"cell_type":"code","source":"KernelVersionDatasetSources = pd.read_csv('/kaggle/input/meta-kaggle/KernelVersionDatasetSources.csv', low_memory = False)\nKernelVersionDatasetSources.head().T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:15:02.244720Z","iopub.execute_input":"2023-07-19T09:15:02.245135Z","iopub.status.idle":"2023-07-19T09:15:07.981226Z","shell.execute_reply.started":"2023-07-19T09:15:02.245103Z","shell.execute_reply":"2023-07-19T09:15:07.979993Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"KernelVersionDatasetSources.nunique()","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:15:07.983050Z","iopub.execute_input":"2023-07-19T09:15:07.983487Z","iopub.status.idle":"2023-07-19T09:15:10.205463Z","shell.execute_reply.started":"2023-07-19T09:15:07.983453Z","shell.execute_reply":"2023-07-19T09:15:10.203663Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### <span style=\"color: #0020FF;\"> 5.5 Analyzing Meta Kaggle KernelVersionKernelSources </span>\n\n***\n\n<span style=\"font-family: 'Lucida Console';\"> KernelVersionKernelSources dataset from Meta Kaggle provides the details like 'Id', 'KernelVersionId', and 'SourceKernelVersionId'. There is 53544 unique SourceKernelVersionId present in the dataset along with 329868 unique KernelVersionId. </span>\n\n***","metadata":{}},{"cell_type":"code","source":"KernelVersionKernelSources = pd.read_csv('/kaggle/input/meta-kaggle/KernelVersionKernelSources.csv', low_memory=False)\nKernelVersionKernelSources.head().T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:15:10.207214Z","iopub.execute_input":"2023-07-19T09:15:10.207593Z","iopub.status.idle":"2023-07-19T09:15:10.767926Z","shell.execute_reply.started":"2023-07-19T09:15:10.207563Z","shell.execute_reply":"2023-07-19T09:15:10.766420Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"KernelVersionKernelSources.nunique()","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:15:10.769971Z","iopub.execute_input":"2023-07-19T09:15:10.770442Z","iopub.status.idle":"2023-07-19T09:15:10.880559Z","shell.execute_reply.started":"2023-07-19T09:15:10.770406Z","shell.execute_reply":"2023-07-19T09:15:10.879295Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### <span style=\"color: #0020FF;\"> 5.6 Analyzing Meta Kaggle KernelVersions </span>\n\n***\n\n<span style=\"font-family: 'Lucida Console';\"> KernelVersions dataset from Meta Kaggle provides the details like 'Id', 'ScriptId', and 'CreationDate' gives a quick reference to choose useful scripts by referring to the forked versions of different scripts based on the lines modified. </span>\n\n***","metadata":{}},{"cell_type":"code","source":"KernelVersions = pd.read_csv('/kaggle/input/meta-kaggle/KernelVersions.csv', low_memory=False)\nKernelVersions.tail().T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:15:10.882881Z","iopub.execute_input":"2023-07-19T09:15:10.884181Z","iopub.status.idle":"2023-07-19T09:16:22.117797Z","shell.execute_reply.started":"2023-07-19T09:15:10.884126Z","shell.execute_reply":"2023-07-19T09:16:22.112826Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### <span style=\"color: #0020FF;\"> 5.7 Analyzing Meta Kaggle KernelVotes </span>\n\n***\n\n<span style=\"font-family: 'Lucida Console';\"> KernelVotes dataset from Meta Kaggle provides the details like 'Id', 'UserId', 'VoteDate' etc. that can provide an insight into the different kernel versions created by the Kaggle users. </span>\n\n***","metadata":{}},{"cell_type":"code","source":"KernelVotes = pd.read_csv('/kaggle/input/meta-kaggle/KernelVotes.csv', low_memory=False)\n# Filtered data for the recent two years\nKernelVotes['VoteDate'] = pd.to_datetime(KernelVotes['VoteDate'])  # Convert to datetime \nKernel_filtered_df = KernelVotes[(KernelVotes['VoteDate'] >= '2021-01-01') & (KernelVotes['VoteDate'] <= '2023-12-31')]\n\nprint(\"The total number of data : \",KernelVotes['VoteDate'].shape[0])\nn_rows = Kernel_filtered_df.shape[0]\nprint(\"The total number of data during the recent two years '21-'23 : \",n_rows)\nKernel_filtered_df.head(n_rows).T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:16:22.124726Z","iopub.execute_input":"2023-07-19T09:16:22.126348Z","iopub.status.idle":"2023-07-19T09:16:53.258757Z","shell.execute_reply.started":"2023-07-19T09:16:22.126228Z","shell.execute_reply":"2023-07-19T09:16:53.257048Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### <span style=\"color: #0020FF;\"> 5.8 Analyzing Meta Kaggle Kernels </span>\n\n***\n\n<span style=\"font-family: 'Lucida Console';\"> The Kernels dataset from Meta Kaggle provides the details like 'AuthorUserId', 'CreationDate' etc. which provides a quick overview of the author, the votes, etc. on the Kaggle platform over the recent two-year timespan. </span>\n\n***","metadata":{}},{"cell_type":"code","source":"Kernels = pd.read_csv('/kaggle/input/meta-kaggle/Kernels.csv', low_memory=False)\nKernels.tail().T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:16:53.260791Z","iopub.execute_input":"2023-07-19T09:16:53.261231Z","iopub.status.idle":"2023-07-19T09:17:00.526137Z","shell.execute_reply.started":"2023-07-19T09:16:53.261194Z","shell.execute_reply":"2023-07-19T09:17:00.524389Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# <span style=\"color: #674AB3;\"> 6. Exploring more on Meta Kaggle </span> 📰\n\n### <span style=\"color: #0020FF;\"> 6.1 Analyzing Meta Kaggle Organizations </span>\n\n***\n\n<span style=\"font-family: 'Lucida Console';\"> Organizations dataset from Meta Kaggle provides the details like 'Id', 'CreationDate' etc. that mostly also include the organizations' intent in conducting the Kaggle competitions. </span>\n\n***","metadata":{}},{"cell_type":"code","source":"Organizations = pd.read_csv('/kaggle/input/meta-kaggle/Organizations.csv', low_memory=False)\n\n# Filtered data for the recent two years\nOrganizations['CreationDate'] = pd.to_datetime(Organizations['CreationDate'])  # Convert to datetime \nOrganizations_filtered_df = Organizations[(Organizations['CreationDate'] >= '2021-01-01') & (Organizations['CreationDate'] <= '2023-12-31')]\n\nprint(\"The total number of data : \",Organizations['CreationDate'].shape[0])\nn_rows = Organizations_filtered_df.shape[0]\nprint(\"The total number of data during the recent two years '21-'23 : \",n_rows)\nOrganizations_filtered_df.head(n_rows).T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:17:00.528261Z","iopub.execute_input":"2023-07-19T09:17:00.528769Z","iopub.status.idle":"2023-07-19T09:17:00.599979Z","shell.execute_reply.started":"2023-07-19T09:17:00.528724Z","shell.execute_reply":"2023-07-19T09:17:00.598463Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### <span style=\"color: #0020FF;\"> 6.2 Analyzing Meta Kaggle Submissions </span>\n\n***\n\n<span style=\"font-family: 'Lucida Console';\"> The submission dataset from Meta Kaggle provides the details like 'SubmittedUserId', 'SubmissionDate' etc. helps one to gain an overview of the different aspects of submissions in competition at Kaggle like the scoring over the recent two-year timeline. </span>\n\n***","metadata":{}},{"cell_type":"code","source":"Submissions = pd.read_csv('/kaggle/input/meta-kaggle/Submissions.csv', low_memory=False)\n\n# Filtered data for the recent two years\nSubmissions['ScoreDate'] = pd.to_datetime(Submissions['ScoreDate'])  # Convert to datetime \nSubmissions_filtered_df = Submissions[(Submissions['ScoreDate'] >= '2021-01-01') & (Submissions['ScoreDate'] <= '2023-12-31')]\n\nprint(\"The total number of data : \",Submissions['ScoreDate'].shape[0])\nn_rows = Submissions_filtered_df.shape[0]\nprint(\"The total number of data during the recent two years '21-'23 : \",n_rows)\nSubmissions_filtered_df.head(n_rows).T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:17:00.601697Z","iopub.execute_input":"2023-07-19T09:17:00.602114Z","iopub.status.idle":"2023-07-19T09:18:26.871355Z","shell.execute_reply.started":"2023-07-19T09:17:00.602078Z","shell.execute_reply":"2023-07-19T09:18:26.869775Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### <span style=\"color: #0020FF;\"> 6.3 Analyzing Meta Kaggle Tags </span>\n\n***\n\n<span style=\"font-family: 'Lucida Console';\"> Tags dataset from Meta Kaggle provides the details like 'Id', 'DatasetCount', 'CompetitionCount', 'KernelCount' etc. to understand the topic area, detailed description, etc. associated with each Id in the Meta Kaggle data updated. </span>\n\n***","metadata":{}},{"cell_type":"code","source":"Tags = pd.read_csv('/kaggle/input/meta-kaggle/Tags.csv', low_memory=False)\nTags.head(Tags.shape[0]).T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:18:26.873240Z","iopub.execute_input":"2023-07-19T09:18:26.873617Z","iopub.status.idle":"2023-07-19T09:18:26.924751Z","shell.execute_reply.started":"2023-07-19T09:18:26.873584Z","shell.execute_reply":"2023-07-19T09:18:26.923505Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### <span style=\"color: #0020FF;\"> 6.4 Analyzing Meta Kaggle TeamMemberships </span>\n\n***\n\n<span style=\"font-family: 'Lucida Console';\"> TeamMemberships dataset from Meta Kaggle provides the details like 'Id', 'UserId', 'RequestDate' etc. giving insights related to the membership requests made by the different teams in the Kaggle platform. </span>\n\n***","metadata":{}},{"cell_type":"code","source":"TeamMemberships = pd.read_csv('/kaggle/input/meta-kaggle/TeamMemberships.csv', low_memory=False)\n\n# Filtered data for the recent two years\nTeamMemberships['RequestDate'] = pd.to_datetime(TeamMemberships['RequestDate'])  # Convert to datetime \nTeamMemberships_filtered_df = TeamMemberships[(TeamMemberships['RequestDate'] >= '2021-01-01') & (TeamMemberships['RequestDate'] <= '2023-12-31')]\n\nprint(\"The total number of data : \",TeamMemberships['RequestDate'].shape[0])\nn_rows = TeamMemberships_filtered_df.shape[0]\nprint(\"The total number of data during the recent two years '21-'23 : \",n_rows)\nTeamMemberships_filtered_df.head(n_rows).T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:18:26.926843Z","iopub.execute_input":"2023-07-19T09:18:26.927237Z","iopub.status.idle":"2023-07-19T09:18:53.884020Z","shell.execute_reply.started":"2023-07-19T09:18:26.927204Z","shell.execute_reply":"2023-07-19T09:18:53.882782Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### <span style=\"color: #0020FF;\"> 6.5 Analyzing Meta Kaggle Teams </span>\n\n***\n\n<span style=\"font-family: 'Lucida Console';\"> Teams dataset from Meta Kaggle provides the details like 'Id', 'Medal', 'LastSubmissionDate' etc. that draws our attention to the teams who have won the medals, the type of competition in which they had become successful, etc. </span>\n\n***","metadata":{}},{"cell_type":"code","source":"Teams = pd.read_csv('/kaggle/input/meta-kaggle/Teams.csv', low_memory=False)\n\n# Filtered data for the recent two years\nTeams['LastSubmissionDate'] = pd.to_datetime(Teams['LastSubmissionDate'])  # Convert to datetime \nTeams_filtered_df = Teams[(Teams['LastSubmissionDate'] >= '2021-01-01') & (Teams['LastSubmissionDate'] <= '2023-12-31')]\n\nprint(\"The total number of data : \",Teams['LastSubmissionDate'].shape[0])\nn_rows = Teams_filtered_df.shape[0]\nprint(\"The total number of data during the recent two years '21-'23 : \",n_rows)\nTeams_filtered_df.head(n_rows).T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:18:53.885788Z","iopub.execute_input":"2023-07-19T09:18:53.886167Z","iopub.status.idle":"2023-07-19T09:19:26.176362Z","shell.execute_reply.started":"2023-07-19T09:18:53.886137Z","shell.execute_reply":"2023-07-19T09:19:26.174681Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### <span style=\"color: #0020FF;\"> 6.6 Analyzing Meta Kaggle User Achievements </span>\n\n***\n\n<span style=\"font-family: 'Lucida Console';\"> UserAchievements dataset from Meta Kaggle provides the details like 'TierAchievementDate', 'AchievementType', etc. that highlights the specific achievement of the Kaggle users.</span>\n\n***","metadata":{}},{"cell_type":"code","source":"UserAchievements = pd.read_csv('/kaggle/input/meta-kaggle/UserAchievements.csv', low_memory=False)\nUserAchievements.tail().T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:26:30.387856Z","iopub.execute_input":"2023-07-19T09:26:30.388467Z","iopub.status.idle":"2023-07-19T09:29:16.450461Z","shell.execute_reply.started":"2023-07-19T09:26:30.388406Z","shell.execute_reply":"2023-07-19T09:29:16.448898Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### <span style=\"color: #0020FF;\"> 6.7 Analyzing Meta Kaggle User Followers </span>\n\n***\n\n<span style=\"font-family: 'Lucida Console';\"> UserFollowers dataset from Meta Kaggle provides the details like 'UserId', 'CreationDate' etc. that provide comprehensive data to interpret the flow of user followers in the Kaggle platform. </span>\n\n***","metadata":{}},{"cell_type":"code","source":"UserFollowers = pd.read_csv('/kaggle/input/meta-kaggle/UserFollowers.csv', low_memory=False)\nUserFollowers.tail().T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:29:20.674684Z","iopub.execute_input":"2023-07-19T09:29:20.675153Z","iopub.status.idle":"2023-07-19T09:29:22.543382Z","shell.execute_reply.started":"2023-07-19T09:29:20.675118Z","shell.execute_reply":"2023-07-19T09:29:22.542031Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### <span style=\"color: #0020FF;\"> 6.8 Analyzing Meta Kaggle User Organizations </span>\n\n<span style=\"font-family: 'Lucida Console';\"> UserOrganizations dataset from Meta Kaggle provides the details like 'UserId', 'JoinDate' etc. that provide comprehensive data to interpret the flow of user followers in the Kaggle platform.</span>\n","metadata":{}},{"cell_type":"code","source":"UserOrganizations = pd.read_csv('/kaggle/input/meta-kaggle/UserOrganizations.csv', low_memory=False)\nUserOrganizations.tail().T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:29:32.302390Z","iopub.execute_input":"2023-07-19T09:29:32.303496Z","iopub.status.idle":"2023-07-19T09:29:32.330751Z","shell.execute_reply.started":"2023-07-19T09:29:32.303443Z","shell.execute_reply":"2023-07-19T09:29:32.329599Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"UserOrganizations['JoinDate'] = pd.to_datetime(UserOrganizations['JoinDate'])\nstart_date = pd.to_datetime('2021-01-01')\nend_date = pd.to_datetime('2023-12-31')\nfiltered_UserOrganizations = UserOrganizations[(UserOrganizations['JoinDate'] >= start_date) & (UserOrganizations['JoinDate'] <= end_date)]\nsorted_UserOrganizations = filtered_UserOrganizations.sort_values(by='JoinDate', ascending=False)\ncounts = sorted_UserOrganizations.value_counts()\nprint(counts)","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:29:35.559609Z","iopub.execute_input":"2023-07-19T09:29:35.560102Z","iopub.status.idle":"2023-07-19T09:29:35.598775Z","shell.execute_reply.started":"2023-07-19T09:29:35.560062Z","shell.execute_reply":"2023-07-19T09:29:35.597333Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### <span style=\"color: #0020FF;\"> 6.9 Analyzing Meta Kaggle Users </span>\n\n<span style=\"font-family: 'Lucida Console';\"> Users dataset from Meta Kaggle provides the details like 'UserId', 'CreationDate' etc. that provide comprehensive data to interpret the flow of user followers in the Kaggle platform. The performance analysis using the dataset has been done in recent years to gain a glimpse into the interesting fact that the challenges to be solved at Kaggle do need perseverance, dedication, and of course skill-up! </span>","metadata":{}},{"cell_type":"code","source":"Users= pd.read_csv('/kaggle/input/meta-kaggle/Users.csv', low_memory=False)\nUsers.tail().T","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:29:47.273976Z","iopub.execute_input":"2023-07-19T09:29:47.274465Z","iopub.status.idle":"2023-07-19T09:31:03.892062Z","shell.execute_reply.started":"2023-07-19T09:29:47.274414Z","shell.execute_reply":"2023-07-19T09:31:03.891024Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Users.describe()","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:32:07.574281Z","iopub.execute_input":"2023-07-19T09:32:07.575578Z","iopub.status.idle":"2023-07-19T09:32:08.565722Z","shell.execute_reply.started":"2023-07-19T09:32:07.575532Z","shell.execute_reply":"2023-07-19T09:32:08.564208Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Sorting Kaggle users based on their performance tier using the Users dataset in recent two year timeline.\n\nUsers['RegisterDate'] = pd.to_datetime(Users['RegisterDate'])\nstart_date = pd.to_datetime('2021-01-01')\nend_date = pd.to_datetime('2023-12-31')\nfiltered_Users = Users[(Users['RegisterDate'] >= start_date) & (Users['RegisterDate'] <= end_date)]\nsorted_Users = filtered_Users.sort_values(by='PerformanceTier', ascending=False)\ncounts = sorted_Users['PerformanceTier'].value_counts()\nprint(counts)\n\n# Create a scatter plot\nplt.figure(figsize=(5, 2))\nplt.scatter(counts.index, counts.values)\nplt.xlabel('Categories')\nplt.ylabel('Count')\nplt.title('Scatter Plot of Performance Tier')\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:32:23.728491Z","iopub.execute_input":"2023-07-19T09:32:23.728997Z","iopub.status.idle":"2023-07-19T09:32:30.829553Z","shell.execute_reply.started":"2023-07-19T09:32:23.728933Z","shell.execute_reply":"2023-07-19T09:32:30.827979Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# <span style=\"color: #674AB3;\">  Conclusion </span> 🙂\n> **\"The more that you read, the more things you will know. The more that you learn, the more places you'll go.\"** <br> \n> **— [Dr. Seuss](https://www.encyclopedia.com/children/academic-and-educational-journals/seuss-dr-1904-1991)**\n\n<span style=\"font-family: 'Lucida Console';\"> From choosing the suitable models to the suitable evaluation metrics for the winning data science projects, the Meta Kaggle dataset provides us with up-to-date insights into the state-of-the-art winning recipes behind the Kaggle competition winning team's success stories. The analytical perspective is in itself telling the story inferred from the datasets explored for the recent two years during the timeline 2021-2023. The analysis has been done to explore the recent trends in the choice made by the Kagglers in the competitions, datasets contributed, and more. The pathway to becoming a Kaggle grandmaster needs one to gain the benefit from a successful Kaggler's journey on the <a href=\"https://en.wikipedia.org/wiki/Kaggle\">Kaggle platform</a>. The Meta Kaggle data is extremely useful in reflecting the same. </span>\n\n***","metadata":{}},{"cell_type":"markdown","source":"<span style=\"font-family: 'Lucida Console';\"> Evaluation of the performance of the machine learning model is an important task in the context of model selection. The top seven evaluation metrics that were found to be the most effective for making the right choice of the suitable model based on its performance have been depicted using the pie chart from the preprocessed Kaggle competitions dataset analyzed before. </span>\n\n**More reading on evaluation:**\n1. [Survey on Evaluating the Performance of Machine Learning Algorithms: Past Contributions and Future Roadmap](https://doi.org/10.1016/B978-0-12-816718-2.00016-6)\n1. [Evaluate and select a machine learning algorithm](https://www.ibm.com/garage/method/practices/reason/evaluate-and-select-machine-learning-algorithm/)","metadata":{}},{"cell_type":"code","source":"# Get the sum of 'Count' column\ntotal_count = Comp_sorted_df['Count'].sum()\n\n# Get the top 7 algorithms based on 'Count' column\ntop_algorithms = Comp_sorted_df.nlargest(7, 'Count')\n\n# Extract the algorithm names and their counts\nalgorithm_names = top_algorithms['EvaluationAlgorithmName']\nalgorithm_counts = top_algorithms['Count']\n\n# Calculate the percentage for each algorithm\nalgorithm_percentages = (algorithm_counts / total_count) * 100\n\n# Plot the pie chart\nplt.figure(figsize=(14, 14))\nplt.pie(algorithm_counts, labels=algorithm_names, autopct=lambda p: f'{p:.1f}% ({p * total_count / 100:.0f})', startangle=90)\nplt.title('Top 7 Evaluation Algorithms')\nplt.axis('equal')  \nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:34:08.899130Z","iopub.execute_input":"2023-07-19T09:34:08.899600Z","iopub.status.idle":"2023-07-19T09:34:09.340873Z","shell.execute_reply.started":"2023-07-19T09:34:08.899565Z","shell.execute_reply":"2023-07-19T09:34:09.339599Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### <span style=\"color: #0020FF;\"> AI advancements in Healthcare during recent years: A short read </span> 🏥\n\n<span style=\"font-family: 'Lucida Console';\"> Healthcare services empowered by the support of Artificial Intelligence have a direct impact on improving the quality of services offered to even remote areas affected adversely by the lack of modern facilities. The success of improving the health-related quality of life through improved diagnostics and personalized treatment plans encourages us to rely on the promise of a better future with the responsible and ethical implementation of AI technologies in healthcare. The b<a href=\"https://www.kaggle.com/competitions?searchQuery=health\">recent Kaggle competitions held in healthcare</a> have contributed towards addressing the challenges in healthcare ML models encouraging further research and innovation in state-of-the-art technological advancements.</span>\n\n**References on Recent advances and Future of AI in Healthcare discussions:**\n\n1. <a href=\"https://doi.org/10.1201/9781003333081\">Agarwal, D., Tripathi, K., & Krishen, K. (Eds.). (2023). Concepts of Artificial Intelligence and its Application in Modern Healthcare Systems (1st ed.). CRC Press.</a> \n1. <a href=\"https://doi.org/10.3390/diagnostics13132196\">Shafi, I.; Fatima, A.; Afzal, H.; Díez, I.d.l.T.; Lipari, V.; Breñosa, J.; Ashraf, I. A Comprehensive Review of Recent Advances in Artificial Intelligence for Dentistry E-Health. Diagnostics 2023, 13, 2196. </a>\n1. <a href=\"https://doi.org/10.3390/electronics10202470\"> Bhatt, D.; Patel, C.; Talsania, H.; Patel, J.; Vaghela, R.; Pandya, S.; Modi, K.; Ghayvat, H. CNN Variants for Computer Vision: History, Architecture, Application, Challenges and Future Scope. Electronics 2021, 10, 2470. </a>\n1. [How Does the AI Bill of Rights Impact Healthcare?](https://healthtechmagazine.net/article/2023/07/ai-bill-of-rights-impact-on-healthcare-perfcon)\n1. [What Types of AI Are Being Used in Healthcare?](https://healthtechmagazine.net/article/2023/07/types-ai-in-healthcare-perfcon)\n1. [The Current State of AI in Healthcare and Where It's Going in 2023](https://healthtechmagazine.net/article/2022/12/ai-healthcare-2023-ml-nlp-more-perfcon)\n1. [AI in Healthcare: Benefits and Challenges in 2023](https://research.aimultiple.com/healthcare-ai/)\n1. [AI in healthcare - statistics & facts](https://www.statista.com/topics/10011/ai-in-healthcare/#editorsPicks)\n1. [FDA has now cleared more than 500 healthcare AI algorithms](https://healthexec.com/topics/artificial-intelligence/fda-has-now-cleared-more-500-healthcare-ai-algorithms)\n1. [Economics of Artificial Intelligence, 2022-2023  : Healthcare](https://www.nber.org/programs-projects/projects-and-centers/8230-economics-artificial-intelligence-2022-2023?page=1&perPage=50)\n1. [Artificial Intelligence in Healthcare and Life Sciences - Intel® AI](https://www.intel.com/content/www/us/en/healthcare-it/artificial-intelligence.html)","metadata":{}},{"cell_type":"markdown","source":"### <span style=\"color: #0020FF;\"> AI advancements in Finances during recent years: A short read </span> 🏦\n\n<span style=\"font-family: 'Lucida Console';\">  From financial management and planning to fraud detection, AI tools play a vital role in the Finance industry. The <a href=\"https://www.kaggle.com/competitions?searchQuery=finance\">recent Kaggle competitions held in finances</a> have contributed towards quantitative finance research by formulating investment strategies based on real-world data, to forecast short-term returns in cryptocurrencies, etc. </span>\n\n**References on Recent advances and Future of AI in Finance  discussions:**\n\n1. <a href=\"https://doi.org/10.48550/arXiv.2107.09051\">AI in Finance: Challenges, Techniques and Opportunities - arXiv:2107.09051 [q-fin.CP]</a> \n1. <a href=\"https://doi.org/10.1007/s10796-021-10186-w\"> Enholm, I.M., Papagiannidis, E., Mikalef, P. et al. Artificial Intelligence and Business Value: a Literature Review. Inf Syst Front 24, 1709–1734 (2022).</a>\n1. <a href=\"https://arxiv.org/ftp/arxiv/papers/2110/2110.11999.pdf\"> Machine Learning in FinanceEmerging Trends and Challenges  </a>\n1. [The future role of AI in finance](https://www.worldfinance.com/markets/the-future-role-of-ai-in-finance)\n1. [29 Examples of AI in Finance](https://builtin.com/artificial-intelligence/ai-finance-banking-applications-companies)\n1. [24 Top AI Statistics And Trends In 2023](https://www.forbes.com/advisor/business/ai-statistics/)\n1. [Utilization of artificial intelligence in the banking sector: a systematic literature review](https://link.springer.com/article/10.1057/s41264-022-00176-7)\n1. <a href=\"https://doi.org/10.1080/23322039.2021.2023262\">Hicham Sadok, Fadi Sakka & Mohammed El Hadi El Maknouzi (2022) Artificial intelligence and bank credit analysis: A review, Cogent Economics & Finance, 10:1, DOI: 10.1080/23322039.2021.2023262</a>","metadata":{}},{"cell_type":"markdown","source":"### <span style=\"color: #0020FF;\"> AI advancements in Computer Vision 👁‍🗨 & Natural Language Processing (NLP) 💬 during recent years: A short read </span> \n\n<span style=\"font-family: 'Lucida Console';\">  AI has brought about a revolutionary change in the way visual and textual information can be handled. The <a href=\"https://www.kaggle.com/competitions?searchQuery=NLP\">recent Kaggle competitions held in NLP </a>to bring about innovations like patient note scoring, identification of toxic comments, providing feedback to student essays, etc. The <a href=\"https://www.kaggle.com/competitions?searchQuery=computer+vision\"> recent Kaggle competitions held in Computer Vision</a> has been focused on bringing about solutions to problems like global warming, landmark recognition, improvize Structure from Motion (SfM), etc. \n</span>\n\n**References on Recent advances and Future of AI in Healthcare discussions:**\n\n1. <a href=\"https://doi.org/10.1007/s11042-022-13428-4\">Khurana, D., Koli, A., Khatter, K. et al. Natural language processing: state of the art, current trends and challenges. Multimed Tools Appl 82, 3713–3744 (2023).</a> \n1. <a href=\"https://doi.org/10.1038/s41746-023-00873-0\">\nMeskó, B., Topol, E.J. The imperative for regulatory oversight of large language models (or generative AI) in healthcare. npj Digit. Med. 6, 120 (2023). </a>\n1. <a href=\"https://ai.meta.com/blog/large-language-model-llama-meta-ai/\"> Introducing LLaMA: A foundational, 65-billion-parameter large language model </a>\n1. [Google Research, 2022 & beyond: Language, vision and generative models](https://ai.googleblog.com/2023/01/google-research-2022-beyond-language.html)\n1. [Concentrations of AI-Related Topics in Research: Natural Language Processing](https://cset.georgetown.edu/publication/concentrations-of-ai-related-topics-in-research-natural-language-processing/)\n1. [Top Recent NLP Research](https://opendatascience.com/top-recent-nlp-research/)\n","metadata":{}},{"cell_type":"markdown","source":"> **\"To acquire knowledge, one must study; but to acquire wisdom, one must observe.\"** <br> \n> **— [Marilyn vos Savant](https://www.encyclopedia.com/history/encyclopedias-almanacs-transcripts-and-maps/marilyn-vos-savant)**\n\n<span style=\"font-family: 'Lucida Console';\"> Knowing about the models and the winning solutions is a guiding light to find the way around the data efficiently to contribute in the best possible way to the problems to be solved using machine learning techniques. The <a href=\"https://www.kaggle.com/WinningModelDocumentationGuidelines\">winning model documentation guidelines</a> specified by Kaggle guarantees the quality of learning that is shared on the Kaggle platform. Last but not least, the <a href=\"https://ourworldindata.org/brief-history-of-ai\">brief history of artificial intelligence</a> in the referenced article provides some relevant depictions of the overall timeline and development of computing machines over the years with a thought to ponder over the future of AI! </span>\n\n**Some More References...**\n\n1. [The state of AI in 2022—and a half decade in review - AI by McKinsey](https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai-in-2022-and-a-half-decade-in-review)\n1. [Winning solutions of kaggle competitions](https://www.kaggle.com/code/sudalairajkumar/winning-solutions-of-kaggle-competitions)\n1. [Data Science Glossary on Kaggle](https://www.kaggle.com/code/shivamb/data-science-glossary-on-kaggle/notebook)\n1. [Getting Started on Kaggle](https://youtu.be/O1P4r0Iy55U)\n1. [How to use Kaggle](https://www.kaggle.com/docs/competitions)\n1. [Kaggle Competitions Guide](https://www.kaggle.com/learn-guide/kaggle-competitions)\n1. [Learnings from Kaggle’s Forecasting Competitions](https://arxiv.org/abs/2009.07701)\n1. [The Most Comprehensive List of Kaggle Solutions and Ideas](https://www.kdnuggets.com/2022/11/comprehensive-list-kaggle-solutions-ideas.html)\n1. [How to enter a competition using Kaggle notebooks?](https://youtu.be/GJBOMWpLpTQ)\n1. [State of Machine Learning and Data Science 2021 Panel, Live at Google Cloud Next | Kaggle](https://youtu.be/5RkUDlHqF_E)\n1. [How to use Kernels and Forums to Win Kaggle Competitions | by Jian Qiao | Kaggle](https://youtu.be/1WB8DSnV6bY)\n1. [How to Win Kaggle Competitions | by Pavel Ostyakov | Kaggle Days Paris](https://youtu.be/tsGGpe-onZI)\n1. [2021 BIPOC Grant Program Project Showcase](https://www.kaggle.com/bipoc2021-project-showcase)\n1. [KaggleX Mentorship Program Project Showcase](https://www.kaggle.com/cohort2-2023-project-showcase)\n1. [Kaggle Winner's Blog - Interviews with top Kagglers by the Kaggle Team](https://medium.com/kaggle-blog)","metadata":{}},{"cell_type":"markdown","source":"#### <span style=\"color: #0060FF;\"> Generating the submission file for <a href=\"https://www.kaggle.com/competitions/2023-kaggle-ai-report/overview\">2023 Kaggle AI Report</a> </span>","metadata":{}},{"cell_type":"code","source":"# Assign values to the variables\nessay_category = 'Kaggle Competitions'\nessay_url = 'https://www.kaggle.com/deepthiaj/meta-kaggle-analytics-exploring-ai-trends-21-23/'\nfeedback1_url = 'https://www.kaggle.com/code/lrsowmya/kaggle-empowering-data-enthusiasts-to-learn-by-do/comments#2342689'\nfeedback2_url = 'https://www.kaggle.com/code/docxian/kaggle-ai-report-access-to-data/comments#2342456'\nfeedback3_url = 'https://www.kaggle.com/code/kingabzpro/from-analysis-to-ai-the-journey-of-tabular-data/comments#2342513'\n\n# Create the submission.csv file\nsubmission_data = {\n    'type': ['essay_category', 'essay_url', 'feedback1_url', 'feedback2_url', 'feedback3_url'],\n    'value': [essay_category, essay_url, feedback1_url, feedback2_url, feedback3_url]\n}\nsubmission_df = pd.DataFrame(submission_data)\n\n# Save the submission dataframe as submission.csv\nsubmission_df.to_csv('submission.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2023-07-19T09:34:24.997258Z","iopub.execute_input":"2023-07-19T09:34:24.997825Z","iopub.status.idle":"2023-07-19T09:34:25.015318Z","shell.execute_reply.started":"2023-07-19T09:34:24.997780Z","shell.execute_reply":"2023-07-19T09:34:25.014335Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> <span style=\"font-family: 'Lucida Console'; color: #0060FF;\"> I hope you liked the read, and please do post your valuable comments and suggestions which would help me improve the content whenever possible. Please do vote for the notebook if you found value in it! Thanks in advance! 🙂  </span>","metadata":{}}]}