{ "id": 324563, "title": "Papers about Deep learning for Source code modeling", "url": "/competitions/AI4Code/discussion/324563", "author_name": "", "post_date": "2022-05-12T09:39:37.636000", "votes": 8, "comment_count": 0, "views": 0, "content": "

'

\n

If you're looking for exciting new research papers on deep learning for source code modeling, you've come to the right place! In this post, we'll take a look at some of the latest and greatest papers on this topic.
\nYou may be wondering why source code modeling is an important task for deep learning.

\n

After all, source code is just a bunch of text, so why not just use traditional methods like bag-of-words or n-grams?

\n

The reason is that source code is very different from regular text. It has a specific structure and syntax that must be respected in order to produce working software. This means that traditional methods are not well suited to modeling it.
\nDeep learning approaches, on the other hand, are able to learn the complex structure of source code and can therefore generate models that are much more accurate than those produced by traditional methods.
\nSo without further ado, let's take a look at some of the best research papers on deep learning for source code modeling!

\n
\n

code2vec: Learning Distributed Representations of Code

\n
\n

TL;DR: Code is first decomposed to a collection of paths in its abstract syntax tree. Then, the network learns the atomic representation of each path while simultaneously learning how to aggregate a set of them.

\n
\n
\n

Software Vulnerability Detection via Deep Learning over Disaggregated Code Graph Representation

\n
\n

TL;DR: We develop a novel graph neural network (GNN) to exploit both the semantic context and structural regularity of a program, in order to improve prediction performance. Compared with a generic GNN, our enhancements include a synthesis of multiple representations learned from the several parsed graphs of a program, and a new training loss metric that leverages the fine granularity of labeling.

\n
\n
\n
\n

An Empirical Study on Vulnerability Detection for Source Code Software based on Deep Learning

\n
\n

*TL;DR: Is BiLSTM the most suitable choice for vulnerability detection? This paper conducted a series of experiments to investigate the effectiveness of different neural network models for source code vulnerability detection.

\n
\n
\n
\n

Automated Source Code Generation and Auto-completion Using Deep Learning: Comparing and Discussing Current Language-Model-Related Approaches

\n
\n

TL;DR: We trained and compared different neural network architectures on a Python dataset for code generation and filling mask tasks.

\n
\n
\n
\n

Deep Learning for Software Vulnerabilities Detection Using Code Metrics

\n
\n

TL;DR: The study found that code metrics are very good but not the best to use as features in deep learning-based vulnerabilities detectors.

\n
\n
\n
\n

Deep Learning for Source Code Modeling and Generation

\n
\n

TL;DR: We provide a comprehensive review of existing deep learning methods for source code modeling and generation, formulate common program learning tasks under an encoder-decoder framework, and introduce recent deep learning mechanisms suitable to solve such problems.

\n
\n
\n
\n

Asteria: Deep Learning-based AST-Encoding for Cross-platform Binary Code Similarity Detection

\n
\n

TL;DR: proposed a deep learning method to improve code similarity detection for security applications

\n
\n
\n
\n

A Simple Approach for Handling Out-of-Vocabulary Identifiers in Deep Learning for Source Code

\n
\n

TL;DR: We propose a simple, yet effective method, based on identifier anonymization, to handle out-of-vocabulary (OOV) identifiers. Our method can be treated as a preprocessing step and, therefore, allows for easy implementation.

\n
\n
\n
\n

A Survey on Software Defect Prediction Using Deep Learning

\n
\n

TL;DR: We survey the recent deep learning techniques for defect prediction and analyze their advantages and disadvantages. We identify the open problems and present the recent trends in the field.

\n
\n
\n
\n

Deep Learning Based Code Generation from Requirements Text: Are We There Yet?

\n
\n

TL;DR: We tried to generate code from requirements using deep learning, but it didn't work out very well.

\n
\n
\n
\n

Multiple Authors Identification from Source Code Using Deep Learning Model

\n
\n

TL;DR: We propose a Deep Learning approach to identify multiple authors of source codes by analyzing the coding style of authors. In this method, we use several machine learning and deep learning-based algorithm i.e. Deep Neural Network(DNN), Support Vector Machine(SVM), and Long short-term memory(LSTM) for detecting multiple authors from source code. The code2seq model over Abstract Syntax Tree is used to identify the structural representation of source code which helps to achieve better performance of the algorithms.

\n
\n
\n
\n

A Survey on Deep Learning-Based Source Code Defect Analysis

\n
\n

TL;DR: Deep learning-based methods can automatically extract features from source code, which avoids errors caused by human experts to a certain extent.

\n
\n
", "messages": [ { "id": 1785633, "postDate": "2022-05-12T09:39:37.637Z", "content": "

'

\n

If you're looking for exciting new research papers on deep learning for source code modeling, you've come to the right place! In this post, we'll take a look at some of the latest and greatest papers on this topic.
\nYou may be wondering why source code modeling is an important task for deep learning.

\n

After all, source code is just a bunch of text, so why not just use traditional methods like bag-of-words or n-grams?

\n

The reason is that source code is very different from regular text. It has a specific structure and syntax that must be respected in order to produce working software. This means that traditional methods are not well suited to modeling it.
\nDeep learning approaches, on the other hand, are able to learn the complex structure of source code and can therefore generate models that are much more accurate than those produced by traditional methods.
\nSo without further ado, let's take a look at some of the best research papers on deep learning for source code modeling!

\n
\n

code2vec: Learning Distributed Representations of Code

\n
\n

TL;DR: Code is first decomposed to a collection of paths in its abstract syntax tree. Then, the network learns the atomic representation of each path while simultaneously learning how to aggregate a set of them.

\n
\n
\n

Software Vulnerability Detection via Deep Learning over Disaggregated Code Graph Representation

\n
\n

TL;DR: We develop a novel graph neural network (GNN) to exploit both the semantic context and structural regularity of a program, in order to improve prediction performance. Compared with a generic GNN, our enhancements include a synthesis of multiple representations learned from the several parsed graphs of a program, and a new training loss metric that leverages the fine granularity of labeling.

\n
\n
\n
\n

An Empirical Study on Vulnerability Detection for Source Code Software based on Deep Learning

\n
\n

*TL;DR: Is BiLSTM the most suitable choice for vulnerability detection? This paper conducted a series of experiments to investigate the effectiveness of different neural network models for source code vulnerability detection.

\n
\n
\n
\n

Automated Source Code Generation and Auto-completion Using Deep Learning: Comparing and Discussing Current Language-Model-Related Approaches

\n
\n

TL;DR: We trained and compared different neural network architectures on a Python dataset for code generation and filling mask tasks.

\n
\n
\n
\n

Deep Learning for Software Vulnerabilities Detection Using Code Metrics

\n
\n

TL;DR: The study found that code metrics are very good but not the best to use as features in deep learning-based vulnerabilities detectors.

\n
\n
\n
\n

Deep Learning for Source Code Modeling and Generation

\n
\n

TL;DR: We provide a comprehensive review of existing deep learning methods for source code modeling and generation, formulate common program learning tasks under an encoder-decoder framework, and introduce recent deep learning mechanisms suitable to solve such problems.

\n
\n
\n
\n

Asteria: Deep Learning-based AST-Encoding for Cross-platform Binary Code Similarity Detection

\n
\n

TL;DR: proposed a deep learning method to improve code similarity detection for security applications

\n
\n
\n
\n

A Simple Approach for Handling Out-of-Vocabulary Identifiers in Deep Learning for Source Code

\n
\n

TL;DR: We propose a simple, yet effective method, based on identifier anonymization, to handle out-of-vocabulary (OOV) identifiers. Our method can be treated as a preprocessing step and, therefore, allows for easy implementation.

\n
\n
\n
\n

A Survey on Software Defect Prediction Using Deep Learning

\n
\n

TL;DR: We survey the recent deep learning techniques for defect prediction and analyze their advantages and disadvantages. We identify the open problems and present the recent trends in the field.

\n
\n
\n
\n

Deep Learning Based Code Generation from Requirements Text: Are We There Yet?

\n
\n

TL;DR: We tried to generate code from requirements using deep learning, but it didn't work out very well.

\n
\n
\n
\n

Multiple Authors Identification from Source Code Using Deep Learning Model

\n
\n

TL;DR: We propose a Deep Learning approach to identify multiple authors of source codes by analyzing the coding style of authors. In this method, we use several machine learning and deep learning-based algorithm i.e. Deep Neural Network(DNN), Support Vector Machine(SVM), and Long short-term memory(LSTM) for detecting multiple authors from source code. The code2seq model over Abstract Syntax Tree is used to identify the structural representation of source code which helps to achieve better performance of the algorithms.

\n
\n
\n
\n

A Survey on Deep Learning-Based Source Code Defect Analysis

\n
\n

TL;DR: Deep learning-based methods can automatically extract features from source code, which avoids errors caused by human experts to a certain extent.

\n
\n
", "rawMarkdown": "'\n\nIf you're looking for exciting new research papers on deep learning for source code modeling, you've come to the right place! In this post, we'll take a look at some of the latest and greatest papers on this topic.\nYou may be wondering why source code modeling is an important task for deep learning. \n\nAfter all, source code is just a bunch of text, so why not just use traditional methods like bag-of-words or n-grams?\n\n\nThe reason is that source code is very different from regular text. It has a specific structure and syntax that must be respected in order to produce working software. This means that traditional methods are not well suited to modeling it.\nDeep learning approaches, on the other hand, are able to learn the complex structure of source code and can therefore generate models that are much more accurate than those produced by traditional methods.\nSo without further ado, let's take a look at some of the best research papers on deep learning for source code modeling!\n\n\n_____\n\n\n**[code2vec: Learning Distributed Representations of Code](https://arxiv.org/abs/1803.09473)**\n\n> **TL;DR:** Code is first decomposed to a collection of paths in its abstract syntax tree. Then, the network learns the atomic representation of each path while simultaneously learning how to aggregate a set of them.\n\n\n_____\n\n\n**[Software Vulnerability Detection via Deep Learning over Disaggregated Code Graph Representation](https://www.semanticscholar.org/paper/3810b17a83be400bd24c595b389c8c8065ee4d42)**\n\n> **TL;DR:** We develop a novel graph neural network (GNN) to exploit both the semantic context and structural regularity of a program, in order to improve prediction performance. Compared with a generic GNN, our enhancements include a synthesis of multiple representations learned from the several parsed graphs of a program, and a new training loss metric that leverages the fine granularity of labeling.\n\n_____\n\n\n\n_____\n**[An Empirical Study on Vulnerability Detection for Source Code Software based on Deep Learning](https://doi.org/10.1109/QRS-C55045.2021.00173)**\n\n> ***TL;DR:** Is BiLSTM the most suitable choice for vulnerability detection? This paper conducted a series of experiments to investigate the effectiveness of different neural network models for source code vulnerability detection.\n\n\n_____\n\n\n\n_____\n\n**[Automated Source Code Generation and Auto-completion Using Deep Learning: Comparing and Discussing Current Language-Model-Related Approaches](https://paperswithcode.com/paper/automated-source-code-generation-and-auto)**\n\n> **TL;DR:** We trained and compared different neural network architectures on a Python dataset for code generation and filling mask tasks.\n\n_____\n\n\n\n_____\n\n**[Deep Learning for Software Vulnerabilities Detection Using Code Metrics](https://doi.org/10.1109/ACCESS.2020.2988557)**\n\n> **TL;DR:** The study found that code metrics are very good but not the best to use as features in deep learning-based vulnerabilities detectors.\n\n_____\n\n\n\n_____\n\n\n**[Deep Learning for Source Code Modeling and Generation](https://paperswithcode.com/paper/deep-learning-for-source-code-modeling-and)**\n\n> **TL;DR:** We provide a comprehensive review of existing deep learning methods for source code modeling and generation, formulate common program learning tasks under an encoder-decoder framework, and introduce recent deep learning mechanisms suitable to solve such problems.\n\n_____\n\n\n\n_____\n\n**[Asteria: Deep Learning-based AST-Encoding for Cross-platform Binary Code Similarity Detection](https://www.semanticscholar.org/paper/b29ba3d5341f8ce595f40279f47f404d2ffe1edb)**\n\n> **TL;DR:** proposed a deep learning method to improve code similarity detection for security applications\n\n_____\n\n\n\n_____\n\n**[A Simple Approach for Handling Out-of-Vocabulary Identifiers in Deep Learning for Source Code](https://paperswithcode.com/paper/a-simple-approach-for-handling-out-of)**\n\n> **TL;DR:** We propose a simple, yet effective method, based on identifier anonymization, to handle out-of-vocabulary (OOV) identifiers. Our method can be treated as a preprocessing step and, therefore, allows for easy implementation.\n\n_____\n\n\n\n_____\n\n\n**[A Survey on Software Defect Prediction Using Deep Learning](https://doi.org/10.3390/MATH9111180)**\n\n> **TL;DR:** We survey the recent deep learning techniques for defect prediction and analyze their advantages and disadvantages. We identify the open problems and present the recent trends in the field.\n\n_____\n\n\n\n_____\n\n**[Deep Learning Based Code Generation from Requirements Text: Are We There Yet?](https://scholar.google.com/scholar?q=Deep%20Learning%20Based%20Code%20Generation%20from%20Requirements%20Text%3A%20Are%20We%20There%20Yet%3F)**\n\n> **TL;DR:** We tried to generate code from requirements using deep learning, but it didn't work out very well.\n\n_____\n\n\n\n_____\n\n**[Multiple Authors Identification from Source Code Using Deep Learning Model](https://doi.org/10.1109/ICECIT54077.2021.9641497)**\n\n> **TL;DR:** We propose a Deep Learning approach to identify multiple authors of source codes by analyzing the coding style of authors. In this method, we use several machine learning and deep learning-based algorithm i.e. Deep Neural Network(DNN), Support Vector Machine(SVM), and Long short-term memory(LSTM) for detecting multiple authors from source code. The code2seq model over Abstract Syntax Tree is used to identify the structural representation of source code which helps to achieve better performance of the algorithms.\n\n_____\n\n\n_____\n\n\n**[A Survey on Deep Learning-Based Source Code Defect Analysis](https://doi.org/10.1109/ICCCS49078.2020.9118556)**\n\n> **TL;DR:** Deep learning-based methods can automatically extract features from source code, which avoids errors caused by human experts to a certain extent.\n\n_____\n\n\n\n", "votes": 8 } ], "comments": [], "raw_markdown_by_id": { "1785633": "'\n\nIf you're looking for exciting new research papers on deep learning for source code modeling, you've come to the right place! In this post, we'll take a look at some of the latest and greatest papers on this topic.\nYou may be wondering why source code modeling is an important task for deep learning. \n\nAfter all, source code is just a bunch of text, so why not just use traditional methods like bag-of-words or n-grams?\n\n\nThe reason is that source code is very different from regular text. It has a specific structure and syntax that must be respected in order to produce working software. This means that traditional methods are not well suited to modeling it.\nDeep learning approaches, on the other hand, are able to learn the complex structure of source code and can therefore generate models that are much more accurate than those produced by traditional methods.\nSo without further ado, let's take a look at some of the best research papers on deep learning for source code modeling!\n\n\n_____\n\n\n**[code2vec: Learning Distributed Representations of Code](https://arxiv.org/abs/1803.09473)**\n\n> **TL;DR:** Code is first decomposed to a collection of paths in its abstract syntax tree. Then, the network learns the atomic representation of each path while simultaneously learning how to aggregate a set of them.\n\n\n_____\n\n\n**[Software Vulnerability Detection via Deep Learning over Disaggregated Code Graph Representation](https://www.semanticscholar.org/paper/3810b17a83be400bd24c595b389c8c8065ee4d42)**\n\n> **TL;DR:** We develop a novel graph neural network (GNN) to exploit both the semantic context and structural regularity of a program, in order to improve prediction performance. Compared with a generic GNN, our enhancements include a synthesis of multiple representations learned from the several parsed graphs of a program, and a new training loss metric that leverages the fine granularity of labeling.\n\n_____\n\n\n\n_____\n**[An Empirical Study on Vulnerability Detection for Source Code Software based on Deep Learning](https://doi.org/10.1109/QRS-C55045.2021.00173)**\n\n> ***TL;DR:** Is BiLSTM the most suitable choice for vulnerability detection? This paper conducted a series of experiments to investigate the effectiveness of different neural network models for source code vulnerability detection.\n\n\n_____\n\n\n\n_____\n\n**[Automated Source Code Generation and Auto-completion Using Deep Learning: Comparing and Discussing Current Language-Model-Related Approaches](https://paperswithcode.com/paper/automated-source-code-generation-and-auto)**\n\n> **TL;DR:** We trained and compared different neural network architectures on a Python dataset for code generation and filling mask tasks.\n\n_____\n\n\n\n_____\n\n**[Deep Learning for Software Vulnerabilities Detection Using Code Metrics](https://doi.org/10.1109/ACCESS.2020.2988557)**\n\n> **TL;DR:** The study found that code metrics are very good but not the best to use as features in deep learning-based vulnerabilities detectors.\n\n_____\n\n\n\n_____\n\n\n**[Deep Learning for Source Code Modeling and Generation](https://paperswithcode.com/paper/deep-learning-for-source-code-modeling-and)**\n\n> **TL;DR:** We provide a comprehensive review of existing deep learning methods for source code modeling and generation, formulate common program learning tasks under an encoder-decoder framework, and introduce recent deep learning mechanisms suitable to solve such problems.\n\n_____\n\n\n\n_____\n\n**[Asteria: Deep Learning-based AST-Encoding for Cross-platform Binary Code Similarity Detection](https://www.semanticscholar.org/paper/b29ba3d5341f8ce595f40279f47f404d2ffe1edb)**\n\n> **TL;DR:** proposed a deep learning method to improve code similarity detection for security applications\n\n_____\n\n\n\n_____\n\n**[A Simple Approach for Handling Out-of-Vocabulary Identifiers in Deep Learning for Source Code](https://paperswithcode.com/paper/a-simple-approach-for-handling-out-of)**\n\n> **TL;DR:** We propose a simple, yet effective method, based on identifier anonymization, to handle out-of-vocabulary (OOV) identifiers. Our method can be treated as a preprocessing step and, therefore, allows for easy implementation.\n\n_____\n\n\n\n_____\n\n\n**[A Survey on Software Defect Prediction Using Deep Learning](https://doi.org/10.3390/MATH9111180)**\n\n> **TL;DR:** We survey the recent deep learning techniques for defect prediction and analyze their advantages and disadvantages. We identify the open problems and present the recent trends in the field.\n\n_____\n\n\n\n_____\n\n**[Deep Learning Based Code Generation from Requirements Text: Are We There Yet?](https://scholar.google.com/scholar?q=Deep%20Learning%20Based%20Code%20Generation%20from%20Requirements%20Text%3A%20Are%20We%20There%20Yet%3F)**\n\n> **TL;DR:** We tried to generate code from requirements using deep learning, but it didn't work out very well.\n\n_____\n\n\n\n_____\n\n**[Multiple Authors Identification from Source Code Using Deep Learning Model](https://doi.org/10.1109/ICECIT54077.2021.9641497)**\n\n> **TL;DR:** We propose a Deep Learning approach to identify multiple authors of source codes by analyzing the coding style of authors. In this method, we use several machine learning and deep learning-based algorithm i.e. Deep Neural Network(DNN), Support Vector Machine(SVM), and Long short-term memory(LSTM) for detecting multiple authors from source code. The code2seq model over Abstract Syntax Tree is used to identify the structural representation of source code which helps to achieve better performance of the algorithms.\n\n_____\n\n\n_____\n\n\n**[A Survey on Deep Learning-Based Source Code Defect Analysis](https://doi.org/10.1109/ICCCS49078.2020.9118556)**\n\n> **TL;DR:** Deep learning-based methods can automatically extract features from source code, which avoids errors caused by human experts to a certain extent.\n\n_____\n\n\n\n" } }