{ "fetched_at": "2026-10-20T06:22:11Z", "source_url": "https://www.kaggle.com/competitions/declutter20/data", "pages": [ { "name": "rules", "content": "This is a page where you can include rules that participants must accept before joining. You may wish to include rules like:\n\n - Don't cheat!\n - Apply yourself!\n - Have fun!", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "data-description", "content": "

This page shows up underneath the data files. Here you describe what files you have provided and the format of each. There is no single format for this page that is appropriate for all competitons, but you should strive to describe as much as you can here. A little time spent describing the data here can save a lot of time answering questions later.

\n

Participants should be able to answer questions like these after reading the data description: What files do I need? What should I expect the data format to be? What am I predicting? What acroynms will I encounter?

\n

File descriptions

\n\n

Data fields

\n* ID\n* type - tagged learned type of comment - Javadoc or line\n* path_to_file\n* begin_line - line comment begins on\n* link_to_comment - link to comment on Github\n* comment - text of the comment\n* non-information/label/Expected - the label for the data - non-informative (True/1/yes)", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Description", "content": "# Decluttering Challenge (DeClutter)\n\nIn the scope of [DocGen2, the Second Software Documentation Generation Challenge](https://dysdoc.github.io/docgen2/index.html), hosted by [ICSME 2020](https://icsme2020.github.io/index.html).\n\n### Task Description\nThe goal of the DeClutter challenge is to build an automated tool that can identify unnecessary software \ndocumentation at the class or file level. For example, a comment saying ```//loop from 1 to N``` just before code implementing \nexactly that is not helpful, and worse, could drift when the underlying code is changed to (N-1) (for example). As such, it is considered as non-informative and labeled as \"Non-information = Yes\". \nThe tool should take in a CSV file in the [described format](https://dysdoc.github.io/docgen2/decluttr-format.html), and output a label for each row as either \"Non-information = Yes\" for clutter, non-informative comments and \"Non-information = No\" for non-clutter ones.\n\n### Definition of Non-information\nIn this study, we label as *Non-information* the comments that contain only non-information sentences. As such, the final criterion for annotation of non-information is defined as follows:\n“non-information is a comment that is completely uninformative and hence useless/should be removed (in the perspective of documentation decluttering)”. \n\nOur definition grounds on the [CADO](https://cado.informatik.uni-hamburg.de/coding-guide/) guidelines for Content Analysis for Software Documentation, which defines as non-information \"any complete sentence or self-contained fragment of text that provides only uninformative boilerplate text\". Among examples, simply restating the name of the API element without adding any detail, or stating the obvious, or simply restating the information in the rest of the code block\" [(Maleej and Robillard, TSE 2013)](https://dl.acm.org/citation.cfm?id=2554417). \n\nOur annotation unit is the entire comment. Consistently with the CADO guidelines, if a comment contains both uninformative and informative sentences, we consider that the uninformative part of the sentence is only context for the informative part, and we rate the comment as 'Non-information = No' (i.e., the comment is informative). \n\n\n### Data Description\nBoth development and test data are distributed as *CSV with UTF-8 encoding*. in the following format \n“id, type, path_to_file, begin_line, link_to_comment, comment, non-information” \n\n#### Sample line\n```\"FR974\";\"Line\";\"https://github.com/nnovielli/jabref/blob/master/src/main/java/org/jabref/gui/maintable/CellFactory.java\";95;\"https://github.com/nnovielli/jabref/blob/master/src/main/java/org/jabref/gui/maintable/CellFactory.java#L95\";\"icon.setToolTipText(printedViewModel.getLocalization());\";\"yes\"```\n\nThe Development dataset [declutter-gold_DevelopmentSet.csv](https://github.com/dysdoc/declutter/blob/master/declutter-gold_DevelopmentSet.csv) contains 1050 rows and the 7 columns described above. \n\n### Evaluation and Ranking\nTools will be ranked according to commonly accepted text classification scoring, e.g. Accuracy, Precision, Recall, Matthews Correlation. We may also award a prize for the tool which does best in some hitherto unappreciated dimension. DeClutter will be run as automatically as possible, using Kaggle as a leaderboard mechanism. This is a great opportunity for new researchers to get involved in the topic.\n\nParticipants (either teams or individuals) will initially have access to the development data only. Later, the unlabelled test data will be released (see the timeline below). After the assessment, the labels for the test data will be released as well. Both development and test data will be extracted from the JabRef project. However, participants might use additional labeled data from other sources in order to develop and implement their systems. Guidelines describing the annotation and providing detailed instructions for participants will be released with the development data.\n\nAfter submitting the predictions for the test data, participants will be required to provide a *short report* including a brief description of their approach, an illustration of their experiments, in particular techniques and resources used (included additional data used for building the model, if any), and an analysis of their results. Formatting guidelines will be released with the test data.\n\n### How to Participate\nWe invite the potential participants to register using the [web form](https://forms.gle/jkGKF44Roc51L6ZT6). **Registration is compulsory to participate in the challenge, although it is not binding**. In any case, independently of your registration, you can decide to withdraw from participating by not sending the system results during the evaluation stage. When registering, you will also have the option to subscribe to the Google group we will create to keep participants up to date with the latest news related to the challenge. Participants are encouraged to share comments and questions with the mailing list. The challenge co-chairs will assist you for any potential issues that could be raised.\n\n### Important Dates\nTimeline for the DeClutter Challenge\n* **15th November 2019:** Development data available to participants\n* **20th April 2020:** Test data available, registration closes\n* **4th May 2020:** System results due to organizers\n* **8th June 2020:** Assessment returned to participants\n\nReport submission deadlines\n* **10th July 2020:** Submission deadline for reports\n* **24th July 2020:** Notification deadline\n* **7th August 2020:** Camera-ready deadline (tentative)\n* **September 2020:** [Workshop](https://dysdoc.github.io/docgen2/index.html) co-located with [ICSME 2020](https://icsme2020.github.io/index.html)\n\n### Challenge Chairs\n[Nicole Novielli](http://collab.di.uniba.it/nicole/), University of Bari, Italy\n\n[Neil Ernst](http://www.neilernst.net/), University of Victoria, Canada\n\n### Contacts\nIf you have any questions, please contact the challenge chairs\n\n\"Creative
The challenge dataset is licensed under a Creative Commons Attribution 4.0 International License.\n\n### References\nIf you have any questions, please contact the challenge chairs\n", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Evaluation", "content": "

The evaluation metric for this competition is Mean F1-Score. The F1 score, commonly used in information retrieval, measures accuracy using the statistics precision p and recall r. Precision is the ratio of true positives (tp) to all predicted positives (tp + fp). Recall is the ratio of true positives to all actual positives (tp + fn). The F1 score is given by:

\n

\\[ F1 = 2\\frac{p \\cdot r}{p+r}\\ \\ \\mathrm{where}\\ \\ p = \\frac{tp}{tp+fp},\\ \\ r = \\frac{tp}{tp+fn} \\]

\n

The F1 metric weights recall and precision equally, and a good retrieval algorithm will maximize both precision and recall simultaneously. Thus, moderately good performance on both will be favored over extremely good performance on one and poor performance on the other.

\n

Submission Format

\n

For every comment in the dataset, submission files should contain two columns: CommentID and Label. Label is Yes if the comment is NonInformation, No if Information.

\n

The file should contain a header and have the following format:

\n```\nCommentID,Label\nFR22,Yes\nFR23,No\nFR456,Yes\n```", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Timeline", "content": "Timeline for the DeClutter Challenge\n* **15th November 2019:** Development data available to participants\n* **20th April 2020:** Test data available, registration closes\n* **4th May 2020:** System results due to organizers\n* **8th June 2020:** Assessment returned to participants\n\nReport submission deadlines\n* **10th July 2020:** Submission deadline for reports\n* **24th July 2020:** Notification deadline\n* **7th August 2020:** Camera-ready deadline (tentative)\n* **September 2020:** [Workshop](https://dysdoc.github.io/docgen2/index.html) co-located with [ICSME 2020](https://icsme2020.github.io/index.html)\n", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false } ] }