Hey everyone!
\nI wanted to start a papers thread and build on it, and hope others share as well, papers to gain domain knowledge. I have no domain knowledge in this area so I have downloaded papers that appeared to be relevant after reading their abstracts. I'll be reading these over the coming days and commenting more as I go through all of them. Hope this helps!
\nCan We Automate Scientific Reviewing? - Arguably the most difficult part of this is defining what a \"good\" review is in the first place, so we first discuss possible evaluation measures for such reviews. We then collect a dataset of papers in the machine learning domain, annotate them with different aspects of content covered in each review, and train targeted summarization models that take in papers to generate reviews. Comprehensive experimental results show that system-generated reviews tend to touch upon more aspects of the paper than human-written reviews, but the generated text can suffer from lower constructiveness for all aspects except the explanation of the core ideas of the papers, which are largely factually correct. We finally summarize eight challenges in the pursuit of a good review generation system together with potential solutions, which, hopefully, will inspire more future research on this subject.
TDMSci: A Specialized Corpus for Scientific Literature Entity Tagging of Tasks Datasets and Metrics - In this paper, we present a new corpus that contains domain expert annotations for Task (T), Dataset (D), Metric (M) entities on 2,000 sentences extracted from NLP papers. We report experiment results on TDM extraction using a simple data augmentation strategy and apply our tagger to around 30,000 NLP papers from the ACL Anthology. The corpus is made publicly available to the community for fostering research on scientific publication summarization (Erera et al., 2019) and knowledge discovery.
Semantic and Relational Spaces in Science of Science: Deep Learning Models for Article Vectorisation - The aim of this paper is to discuss the potential and limits of deep learning for gathering insights about scientific research articles. We focus on document-level embeddings based on the semantic and relational aspects of articles, using Natural Language Processing (NLP) and Graph Neural Networks (GNNs). We explore the different outcomes generated by those techniques. Our results show that using NLP we can encode a semantic space of articles, while with GNN we are able to build a relational space where the social practices of a research community are also encoded.
Deep Learning in Science - This paper provides insights on the diffusion and impact of DL in science. Through a Natural Language Processing (NLP) approach on the arXiv.org publication corpus, we delineate the emerging DL technology and identify a list of relevant search terms.
Navigating the landscape of COVID-19 research through literature analysis: A bird's eye view - In this study we analyze the LitCovid collection, 13,369 COVID-19 related articles found in PubMed as of May 15th, 2020 with the purpose of examining the landscape of literature and presenting it in a format that facilitates information navigation and understanding. We do that by applying state-of-the-art named entity recognition, classification, clustering and other NLP techniques. By applying NER tools, we capture relevant bioentities (such as diseases, internal body organs, etc.) and assess the strength of their relationship with COVID-19 by the extent they are discussed in the corpus.
Drug Repurposing for Cancer: An NLP Approach to Identify Low-Cost Therapies - Our primary contribution is to define the natural language processing pipeline required to obtain such evidence, comprising the following modules: querying, filtering, cancer type entity extraction, therapeutic association classification, and study type classification. Using the subject matter expertise on our team, we create our own datasets for these specialized domain-specific tasks. We obtain promising performance in each of the modules by utilizing modern language modeling techniques and plan to treat them as baseline approaches for future improvement of individual components.
SciBERT: A Pretrained Language Model for Scientific Text - Obtaining large-scale annotated data for NLP tasks in the scientific domain is challenging and expensive. We release SciBERT, a pretrained language model based on BERT (Devlin et al., 2018) to address the lack of high-quality, large-scale labeled scientific data. SciBERT leverages unsupervised pretraining on a large multi-domain corpus of scientific publications to improve performance on downstream scientific NLP tasks. We evaluate on a suite of tasks including sequence tagging, sentence classification and dependency parsing, with datasets from a variety of scientific domains. We demonstrate statistically significant improvements over BERT and achieve new state-of-the-art results on several of these tasks.
Hey everyone!
\nI wanted to start a papers thread and build on it, and hope others share as well, papers to gain domain knowledge. I have no domain knowledge in this area so I have downloaded papers that appeared to be relevant after reading their abstracts. I'll be reading these over the coming days and commenting more as I go through all of them. Hope this helps!
\nCan We Automate Scientific Reviewing? - Arguably the most difficult part of this is defining what a \"good\" review is in the first place, so we first discuss possible evaluation measures for such reviews. We then collect a dataset of papers in the machine learning domain, annotate them with different aspects of content covered in each review, and train targeted summarization models that take in papers to generate reviews. Comprehensive experimental results show that system-generated reviews tend to touch upon more aspects of the paper than human-written reviews, but the generated text can suffer from lower constructiveness for all aspects except the explanation of the core ideas of the papers, which are largely factually correct. We finally summarize eight challenges in the pursuit of a good review generation system together with potential solutions, which, hopefully, will inspire more future research on this subject.
TDMSci: A Specialized Corpus for Scientific Literature Entity Tagging of Tasks Datasets and Metrics - In this paper, we present a new corpus that contains domain expert annotations for Task (T), Dataset (D), Metric (M) entities on 2,000 sentences extracted from NLP papers. We report experiment results on TDM extraction using a simple data augmentation strategy and apply our tagger to around 30,000 NLP papers from the ACL Anthology. The corpus is made publicly available to the community for fostering research on scientific publication summarization (Erera et al., 2019) and knowledge discovery.
Semantic and Relational Spaces in Science of Science: Deep Learning Models for Article Vectorisation - The aim of this paper is to discuss the potential and limits of deep learning for gathering insights about scientific research articles. We focus on document-level embeddings based on the semantic and relational aspects of articles, using Natural Language Processing (NLP) and Graph Neural Networks (GNNs). We explore the different outcomes generated by those techniques. Our results show that using NLP we can encode a semantic space of articles, while with GNN we are able to build a relational space where the social practices of a research community are also encoded.
Deep Learning in Science - This paper provides insights on the diffusion and impact of DL in science. Through a Natural Language Processing (NLP) approach on the arXiv.org publication corpus, we delineate the emerging DL technology and identify a list of relevant search terms.
Navigating the landscape of COVID-19 research through literature analysis: A bird's eye view - In this study we analyze the LitCovid collection, 13,369 COVID-19 related articles found in PubMed as of May 15th, 2020 with the purpose of examining the landscape of literature and presenting it in a format that facilitates information navigation and understanding. We do that by applying state-of-the-art named entity recognition, classification, clustering and other NLP techniques. By applying NER tools, we capture relevant bioentities (such as diseases, internal body organs, etc.) and assess the strength of their relationship with COVID-19 by the extent they are discussed in the corpus.
Drug Repurposing for Cancer: An NLP Approach to Identify Low-Cost Therapies - Our primary contribution is to define the natural language processing pipeline required to obtain such evidence, comprising the following modules: querying, filtering, cancer type entity extraction, therapeutic association classification, and study type classification. Using the subject matter expertise on our team, we create our own datasets for these specialized domain-specific tasks. We obtain promising performance in each of the modules by utilizing modern language modeling techniques and plan to treat them as baseline approaches for future improvement of individual components.
SciBERT: A Pretrained Language Model for Scientific Text - Obtaining large-scale annotated data for NLP tasks in the scientific domain is challenging and expensive. We release SciBERT, a pretrained language model based on BERT (Devlin et al., 2018) to address the lack of high-quality, large-scale labeled scientific data. SciBERT leverages unsupervised pretraining on a large multi-domain corpus of scientific publications to improve performance on downstream scientific NLP tasks. We evaluate on a suite of tasks including sequence tagging, sentence classification and dependency parsing, with datasets from a variety of scientific domains. We demonstrate statistically significant improvements over BERT and achieve new state-of-the-art results on several of these tasks.
Thanks @crained - here is another I was just reading and thought of your list (it deals specifically with finding dataset mentions in papers:
\nhttps://link.springer.com/chapter/10.1007/978-3-030-50417-5_23
", "rawMarkdown": "Thanks @crained - here is another I was just reading and thought of your list (it deals specifically with finding dataset mentions in papers:\n\n[https://link.springer.com/chapter/10.1007/978-3-030-50417-5_23](url)\n", "votes": 1 }, { "id": 1250298, "postDate": "2021-03-23T23:27:55.080Z", "content": "Just read \"Can We Automate Scientific Reviewing?\" last week. Amazing research work. Their evaluation methods are tied with review metrics in many top-tier conferences.
", "rawMarkdown": "Just read \"Can We Automate Scientific Reviewing?\" last week. Amazing research work. Their evaluation methods are tied with review metrics in many top-tier conferences. ", "votes": 2 }, { "id": 1348350, "postDate": "2021-06-14T02:04:01.730Z", "content": "Excellent, thank you very much for sharing…💪👍
", "rawMarkdown": "Excellent, thank you very much for sharing...💪👍" }, { "id": 1348167, "postDate": "2021-06-13T19:41:53.217Z", "content": "Thanks @crained for sharing this. I read and review quite a few papers, so I dont think we are anywhere near reviewing it using an AI. There is so much to be done.
\nThanks
Thanks @crained for sharing!
", "rawMarkdown": "Thanks @crained for sharing!" } ], "comments": [ { "id": 1277923, "author_name": "Mark P", "author_url": "", "post_date": "2021-04-19T11:27:21.997000", "content": "Thanks @crained - here is another I was just reading and thought of your list (it deals specifically with finding dataset mentions in papers:
\nhttps://link.springer.com/chapter/10.1007/978-3-030-50417-5_23
", "votes": 1, "replies": [] }, { "id": 1250298, "author_name": "ericngo", "author_url": "", "post_date": "2021-03-23T23:27:55.080000", "content": "Just read \"Can We Automate Scientific Reviewing?\" last week. Amazing research work. Their evaluation methods are tied with review metrics in many top-tier conferences.
", "votes": 2, "replies": [] }, { "id": 1348350, "author_name": "Henry Javier", "author_url": "", "post_date": "2021-06-14T02:04:01.730000", "content": "Excellent, thank you very much for sharing…💪👍
", "votes": 0, "replies": [] }, { "id": 1348167, "author_name": "KritiDoneria", "author_url": "", "post_date": "2021-06-13T19:41:53.217000", "content": "Thanks @crained for sharing this. I read and review quite a few papers, so I dont think we are anywhere near reviewing it using an AI. There is so much to be done.
\nThanks
Thanks @crained for sharing!
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1250282": "Hey everyone! \n\nI wanted to start a papers thread and build on it, and hope others share as well, papers to gain domain knowledge. I have no domain knowledge in this area so I have downloaded papers that appeared to be relevant after reading their abstracts. I'll be reading these over the coming days and commenting more as I go through all of them. Hope this helps!\n\n- [Can We Automate Scientific Reviewing?](https://arxiv.org/abs/2102.00176) - Arguably the most difficult part of this is defining what a \"good\" review is in the first place, so we first discuss possible evaluation measures for such reviews. We then collect a dataset of papers in the machine learning domain, annotate them with different aspects of content covered in each review, and train targeted summarization models that take in papers to generate reviews. Comprehensive experimental results show that system-generated reviews tend to touch upon more aspects of the paper than human-written reviews, but the generated text can suffer from lower constructiveness for all aspects except the explanation of the core ideas of the papers, which are largely factually correct. We finally summarize eight challenges in the pursuit of a good review generation system together with potential solutions, which, hopefully, will inspire more future research on this subject.\n\n- [TDMSci: A Specialized Corpus for Scientific Literature Entity Tagging of Tasks Datasets and Metrics](https://arxiv.org/abs/2101.10273) - In this paper, we present a new corpus that contains domain expert annotations for Task (T), Dataset (D), Metric (M) entities on 2,000 sentences extracted from NLP papers. We report experiment results on TDM extraction using a simple data augmentation strategy and apply our tagger to around 30,000 NLP papers from the ACL Anthology. The corpus is made publicly available to the community for fostering research on scientific publication summarization (Erera et al., 2019) and knowledge discovery.\n\n- [Semantic and Relational Spaces in Science of Science: Deep Learning Models for Article Vectorisation](https://arxiv.org/abs/2011.02887) - The aim of this paper is to discuss the potential and limits of deep learning for gathering insights about scientific research articles. We focus on document-level embeddings based on the semantic and relational aspects of articles, using Natural Language Processing (NLP) and Graph Neural Networks (GNNs). We explore the different outcomes generated by those techniques. Our results show that using NLP we can encode a semantic space of articles, while with GNN we are able to build a relational space where the social practices of a research community are also encoded.\n\n- [Deep Learning in Science](https://arxiv.org/abs/2009.01575) - This paper provides insights on the diffusion and impact of DL in science. Through a Natural Language Processing (NLP) approach on the arXiv.org publication corpus, we delineate the emerging DL technology and identify a list of relevant search terms. \n\n- [Navigating the landscape of COVID-19 research through literature analysis: A bird's eye view](https://arxiv.org/abs/2008.03397) - In this study we analyze the LitCovid collection, 13,369 COVID-19 related articles found in PubMed as of May 15th, 2020 with the purpose of examining the landscape of literature and presenting it in a format that facilitates information navigation and understanding. We do that by applying state-of-the-art named entity recognition, classification, clustering and other NLP techniques. By applying NER tools, we capture relevant bioentities (such as diseases, internal body organs, etc.) and assess the strength of their relationship with COVID-19 by the extent they are discussed in the corpus. \n\n- [Drug Repurposing for Cancer: An NLP Approach to Identify Low-Cost Therapies](https://arxiv.org/abs/1911.07819) - Our primary contribution is to define the natural language processing pipeline required to obtain such evidence, comprising the following modules: querying, filtering, cancer type entity extraction, therapeutic association classification, and study type classification. Using the subject matter expertise on our team, we create our own datasets for these specialized domain-specific tasks. We obtain promising performance in each of the modules by utilizing modern language modeling techniques and plan to treat them as baseline approaches for future improvement of individual components.\n\n- [SciBERT: A Pretrained Language Model for Scientific Text](https://arxiv.org/abs/1903.10676) - Obtaining large-scale annotated data for NLP tasks in the scientific domain is challenging and expensive. We release SciBERT, a pretrained language model based on BERT (Devlin et al., 2018) to address the lack of high-quality, large-scale labeled scientific data. SciBERT leverages unsupervised pretraining on a large multi-domain corpus of scientific publications to improve performance on downstream scientific NLP tasks. We evaluate on a suite of tasks including sequence tagging, sentence classification and dependency parsing, with datasets from a variety of scientific domains. We demonstrate statistically significant improvements over BERT and achieve new state-of-the-art results on several of these tasks.", "1277923": "Thanks @crained - here is another I was just reading and thought of your list (it deals specifically with finding dataset mentions in papers:\n\n[https://link.springer.com/chapter/10.1007/978-3-030-50417-5_23](url)\n", "1250298": "Just read \"Can We Automate Scientific Reviewing?\" last week. Amazing research work. Their evaluation methods are tied with review metrics in many top-tier conferences. ", "1348350": "Excellent, thank you very much for sharing...💪👍", "1348167": "Thanks @crained for sharing this. I read and review quite a few papers, so I dont think we are anywhere near reviewing it using an AI. There is so much to be done.\nThanks", "1250541": "", "1251727": "Thanks @crained for sharing!" } }