","metadata":{}},{"cell_type":"markdown","source":"# Overview\n\nCiting the competition's hosts:\n> The objective of the competition is to identify the mention of datasets within scientific publications.\n\nThus, given a set of publications, for which we'll have acces to information such as their title, paragraphs titles and text bodies, we'll have to extract short excerpts from the publications that appear to note a dataset. Such work would prove to be greatly beneficial in the context of data sharing and availability. One would be able to quickly search publications of interest that utilize a given dataset in order to gain insights on the dataset, or in the opposite route, quickly figure out what kind of data is being used in a set of un-explored publications that tackle a specific topic of interest. Consequently, publication processing could be automated and research work greatly accelerated.\n\nIn this notebook, we'll take a look at an Exploratory Data Analysis of the training data provided for this competition, as well as building and running a naïve solution that basically performs dataset title string matching (from a set of known datasets titles) to predict whether a given publication sites, or not, a given dataset.\n\n\n### Outline:\n\n1. [Setup and Basic EDA](#head-1) \n 1.1. [Dataset Title VS Dataset Label](#head-1-1) \n 1.2. [Datasets Popularity](#head-1-2) \n 1.3. [Datasets Occurence Together](#head-1-3)\n2. [Wordcloud of the Articles Titles](#head-2) \n 2.1. [Wordclouds of Article Titles by Dataset](#head-2-1)\n3. [Loading JSON Contents into a Pandas DataFrame](#head-3) \n4. [A Naïve Dataset Title Matching Submission](#head-4) ","metadata":{}},{"cell_type":"markdown","source":"# 1. Setup and Basic EDA ","metadata":{}},{"cell_type":"code","source":"import os\nimport re\nimport json\nimport glob\nfrom collections import defaultdict\n\nimport numpy as np\nimport pandas as pd\n\nimport matplotlib.pyplot as plt\nimport plotly.express as px\nimport plotly.graph_objects as go\n\n%matplotlib inline\n\nos.listdir('/kaggle/input/coleridgeinitiative-show-us-the-data')","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We are provided with 4 main pieces of data:\n* `train.csv`: The CSV file containing all the metadata of the publications, such as their title and the dataset they utilize.\n* `train`: The directory containing the actual publications that are referenced in `train.csv` in JSON format.\n* `test`: The directory containing the actual publications that will be used for testing purposes (thus, with no ground truth CSV file available).\n* `sample_submission.csv`: The CSV file containing all the publications IDs in the test set, for which we'll have to populate the prediction column.","metadata":{}},{"cell_type":"code","source":"train = pd.read_csv('../input/coleridgeinitiative-show-us-the-data/train.csv')\ntrain","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The training data contains 19,661 rows, with 5 columns describing each row.","metadata":{}},{"cell_type":"code","source":"train.info()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"That's great! There are no missing values, and the dataset looks complete.","metadata":{}},{"cell_type":"code","source":"for col in train.columns:\n print(f\"{col}: {len(train[col].unique())}\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"It looks like there are only **14,316** unique IDs in the dataset, meaning that some publications include a multitude of datasets. Also, notice that the `pub_title` unique count is slightly smaller than the `Id` unique counts. This points to the precense of several occurences of having 2 separate publications, eahc with a unique ID, but sharing the exact same title.\n\nAlso, there are a total of **45** unique `dataset_title` and 130 unique `dataset_label`. Meaning that a single dataset could have multible labels throughout different publications.","metadata":{}},{"cell_type":"markdown","source":"## 1.1. Dataset Title VS Dataset Label ","metadata":{}},{"cell_type":"code","source":"print(\"Printing below the dataset titles that have multiple dataset labels associated with them:\\n\")\ndatasets_titles_unique = train[\"dataset_title\"].unique()\nfor dataset_title in datasets_titles_unique:\n if len(train[train[\"dataset_title\"] == dataset_title][\"dataset_label\"].unique()) > 1:\n print(f\"'{dataset_title}':\", list(train[train[\"dataset_title\"] == dataset_title][\"dataset_label\"].unique()), \"\\n\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 1.2. Datasets Popularity ","metadata":{}},{"cell_type":"code","source":"dataset_titles_counts = train['dataset_title'].value_counts()\n\nfig = go.Figure(data=[go.Table(\n columnwidth = [0.25, 2, 0.5],\n header=dict(\n values=[\"Rank\", \"Dataset Title\", \"Mentions\"],\n line_color='darkslategray',\n fill_color=\"royalblue\",\n align='center',\n font=dict(color='white', size=12)\n ),\n cells=dict(\n values=np.array([np.array((str(i+1), \"\" + x + \"\", \"\" + str(y) + \"\", )) for i, (x, y) in enumerate(zip(dataset_titles_counts.index, dataset_titles_counts.values))]).T,\n line_color='darkslategray',\n # 2-D list of colors for alternating rows\n fill_color = [[\"white\",\"lavender\"]*25],\n align = 'center',\n font = dict(color = 'darkslategray', size = 11)\n ))\n])\n\nfig.update_layout(\n title={\"text\": \"Datasets Titles Mentions Counts\",\n \"x\": 0.5,\n \"xanchor\":\"center\",\n \"font_size\": 22},\n margin={\"r\":20, \"l\":20})\n\nfig.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 1.3. Datasets Occurence Together ","metadata":{}},{"cell_type":"code","source":"multi_mentions = [] # list of tuples (Id, num_unique_mentions) \n\nfor Id in train.Id.unique():\n num_unique_mentions = len(train[train['Id'] == Id]['dataset_title'].unique())\n if num_unique_mentions > 1:\n multi_mentions.append((Id, num_unique_mentions))\n\nprint(f\"There are {len(multi_mentions)} publications in the training set that mention more than 1 dataset. That is {len(multi_mentions) / len(train.Id.unique()) * 100:.2f}% of the publications.\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"co_mentions = defaultdict(int)\n\nfor Id, num_mentions in multi_mentions:\n co_mentions[tuple(sorted(train[train['Id'] == Id][\"dataset_title\"].unique()))] += 1\n\nprint(f\"There are {len(co_mentions)} unique sets of co-occurences of dataset mentions.\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"co_mentions = dict(sorted(co_mentions.items(), key=lambda item: item[1], reverse=True))\n\nfig = go.Figure(data=[go.Table(\n columnwidth = [0.25, 2, 0.5],\n header=dict(\n values=[\"Rank\", \"Datasets Co-Mentions Sets\", \"Occurences\"],\n line_color='darkslategray',\n fill_color=\"royalblue\",\n align='center',\n font=dict(color='white', size=12)\n ),\n cells=dict(\n values=np.array([np.array((str(i+1), \"\" + str(k) + \"\", \"\" + str(v) + \"\", )) for i, (k, v) in enumerate(co_mentions.items())]).T,\n line_color='darkslategray',\n fill_color = [[\"white\",\"lavender\"]*60],\n align = 'center',\n font = dict(color = 'darkslategray', size = 11)\n ))\n])\n\nfig.update_layout(\n title={\"text\": \"Datasets Mentioned in the same Publications\",\n \"x\": 0.5,\n \"xanchor\":\"center\",\n \"font_size\": 22},\n margin={\"r\":20, \"l\":20})\n\nfig.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 2. Wordcloud of the Articles Titles ","metadata":{}},{"cell_type":"code","source":"from wordcloud import WordCloud, STOPWORDS\n\nwords_in_titles = list(train.pub_title.str.split(expand=True).stack())\n\nwordcloud = WordCloud(stopwords = STOPWORDS,\n background_color = \"white\",\n width = 3000,\n height = 2000\n ).generate(' '.join(words_in_titles))\nplt.figure(1, figsize = (18, 12))\nplt.imshow(wordcloud)\nplt.axis('off')\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 2.1. Wordclouds of Article Titles by Dataset Mentions ","metadata":{}},{"cell_type":"code","source":"words_in_titles_by_dataset = defaultdict(list)\n\n# Separating out positive and negative words (i.e., words appearing in negative and positive tweets),\n# in order to visualize each set of words independently\nfor _, row in train.iterrows():\n words_in_titles_by_dataset[row['dataset_title']].extend(row['pub_title'].split())\n\n# Defining our word cloud drawing function\ndef wordcloud_draw(data, color = 'white'):\n wordcloud = WordCloud(stopwords = STOPWORDS,\n background_color = color,\n width = 3000,\n height = 2000\n ).generate(' '.join(data))\n plt.figure(1, figsize = (12, 8))\n plt.imshow(wordcloud)\n plt.axis('off')\n plt.show()\n\nfor dataset_title in train['dataset_title'].unique():\n print(\"Wordcloud for publications mentioning\", dataset_title, \":\")\n wordcloud_draw(words_in_titles_by_dataset[dataset_title])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 3. Loading JSON Contents into a Pandas DataFrame ","metadata":{}},{"cell_type":"code","source":"# Gathering the files paths\ntrain_files = glob.glob(\"../input/coleridgeinitiative-show-us-the-data/train/*.json\")\ntest_files = glob.glob(\"../input/coleridgeinitiative-show-us-the-data/test/*.json\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Generate the training publications dataframe\ndf_train_publications = pd.DataFrame()\n\nfor train_file in train_files:\n file_data = pd.read_json(train_file)\n file_data.insert(0,'pub_id', train_file.split('/')[-1].split('.')[0])\n df_train_publications = pd.concat([df_train_publications, file_data])\n\ndf_train_publications.to_csv(\"df_train_publications.csv\",index=False)\n\ndf_train_publications","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Generate the testing publications dataframe\ndf_test_publications = pd.DataFrame()\n\nfor test_file in test_files:\n file_data = pd.read_json(test_file)\n file_data.insert(0,'pub_id', test_file.split('/')[-1].split('.')[0])\n df_test_publications = pd.concat([df_test_publications, file_data])\n\ndf_test_publications.to_csv(\"df_test_publications.csv\",index=False)\n\ndf_test_publications","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 4. A Naïve Dataset Title Matching Submission ","metadata":{}},{"cell_type":"markdown","source":"Obviously, the end goal of such a competition is not simply do string matching of known datasets names in order to detect mentions of datasets in publication, however, it is to build a strong enough NLP model that can infer from context whether or not a piece of text in a publication is refering to the usage of a dataset or not.\n\nThat being said, below we will implement a very simple known-dataset string names matching technique as a POC and template for building a submission. Such a technique would be rather useless when applied on publications mentioning datasets not present in our \"known datasets\" list, which is the case in the majority of the hidden test set of this competition.","metadata":{}},{"cell_type":"code","source":"def clean_text(txt):\n return re.sub('[^A-Za-z0-9]+', ' ', str(txt).lower()).strip()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission_df = pd.read_csv('../input/coleridgeinitiative-show-us-the-data/sample_submission.csv', index_col=0)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission_df","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission_df = pd.read_csv('../input/coleridgeinitiative-show-us-the-data/sample_submission.csv', index_col=0)\ndatasets_titles = [x.lower() for x in set(train['dataset_title'].unique()).union(set(train['dataset_label'].unique()))]\n\nlabels = []\nfor index in submission_df.index:\n publication_text = df_test_publications[df_test_publications['pub_id'] == index].text.str.cat(sep='\\n').lower()\n label = []\n for dataset_title in datasets_titles:\n if dataset_title in publication_text:\n label.append(clean_text(dataset_title))\n labels.append('|'.join(label))\n\nsubmission_df['PredictionString'] = labels\n\nsubmission_df.to_csv('submission.csv')\n\nsubmission_df","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# This notebook is under development 🚧\n\n---\n\n## Please upvote if you found it useful 😊","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}
![](\"https://coleridgeinitiative.org/wp-content/uploads/2021/02/rich-context.png\"/)
\n