{
  "fetched_at": "2026-10-19T23:53:05Z",
  "source_url": "https://www.kaggle.com/competitions/flares-subtask-2-5w1hs-classification/data",
  "pages": [
    {
      "name": "data-description",
      "content": "The dataset contains approximately 9,034 5W1H annotations distributed in 190 news items. The 5W1H dataset has been splitted into a proportion of 70% (6,934 5W1H annotation) for training and 30% (2,100 5W1H annotation) for testing. The corpus contains text in Spanish manually annotated with an extraction technique known as 5W1H, which is a journalistic technique consisting of annotating all the entities present in a text related to the questions What, Who, Where, When, Why and How. It was annotated by three expert annotators (2 linguistics and 1 sociologist), all three specialized in NLP and with knowledge in the annotation guideline created for this purpose.\n\nThe language chosen is Spanish because, although it is the fourth most widely spoken language in the world and the second mother tongue in the world in terms of number of speakers, there are few corpora built in Spanish to address the automatic reliability detection task in NLP. \n\nWith regard to the domain, the topics covered in this dataset include economy, sports, science, education, health, society, entertainment, politics and security.\n\nRegarding the collection procedure, 190 news was collected manually and via web crawling. The news was collected from digital newspapers such as ABC, BBC News, CNN Spanish, El Espectador, El Financiero, El Mundo, El País, Huffpost, Marca, La Jornada, El Diestro, Eje21, Periodista Digital, Ok Diario, 20 Minutos, la Vanguardia, among others. For the construction of the dataset, news that did not follow the language, format, extension, or other semantic characteristics desired in the corpus was filtered out.\n\nFiles\n5w1h_subtask_2_train.json - the training set\n5w1h_subtask_2_test.json - the test set",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    },
    {
      "name": "Description",
      "content": "**Subtask 2: 5W1H-based reliability** (approx 9,034 annotations): For each 5W1H detected, participants will have to determine if the language used in each item is reliable or unreliable, following the RUN-AS guideline.\n\nAlong with these labels, the attribute “reliability” will be used to classify reliability in language with the values “Reliable” and “Unreliable”, depending on the two following criteria:\n\n1. Accuracy of the content (which takes into account aspects such as vagueness, ambiguity, lack of evidence or orthotypography). \n     - Example: “A long time ago” (inaccurate = unreliable) vs. “On Friday 19 march” (accurate = reliable)\n     - Example: “A scientist” (inaccurate = unreliable) vs. “The European Medicines Agency” (accurate = reliable)\n\n2. Neutrality of the content (which considers personal remarks, emotionally charged content, quotes, author’s stance or the objectivity of the title)\n     - Example: “In my opinion” (subjective = unreliable) vs. “According to the rector of the University of Alicante” (objective = reliable)\n     - Example: “This news item can save your life” (subjective = unreliable) vs. “This news item talks about the positive effects” (objective = reliable)\n\n---------\nSchedule:\n- February 5, 2024: Call for Participation and participant's subscription\n- March 20, 2024: Kaggle competitions open and Training Set Available\n- May 3, 2024: Testing Set Available\n- May 17, 2024: Submission of Run\n- May 24, 2024: Notification of Results\n- June 14, 2024: Submission of Working Notes by Participants\n- June 28, 2024: Reviews to Participants\n- July 11, 2024: Camera-ready Submissions Due\n\n----------\nParticipant's subscription:\n \nTo participate and receive notifications fill the following Google form ([FLARES Registration](https://docs.google.com/forms/d/e/1FAIpQLSeDE8tPqJtEJI4zM4wVFK06oZ8HYnHNVnj0k1A-74DRrIHQgw/viewform))\n",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    },
    {
      "name": "Evaluation",
      "content": "**For Subtask 2**, focusing on a classification task, the proposed evaluation metrics include: \n\nAccuracy, which measures the proportion of correctly classified news items. \n\nPrecision and Recall assess the model's ability to correctly classify true positives against false positives and capture all actual positives, respectively. \n\nThe F1-Score combines precision and recall, providing a harmonic mean that balances both metrics.",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    },
    {
      "name": "rules",
      "content": "It is allowed to use any algorithm, model, etc, as well as external data.",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    },
    {
      "name": "abstract",
      "content": "FLARES: Fine-Grained Language-based Reliability Detection in Spanish News\n\nDescription of the task:\n\nAssessing the reliability of the language used in news writing is becoming increasingly crucial in today's digital media landscape. Identifying specific segments of a news article to gauge the linguistic credibility offers a more nuanced understanding of the message's truthfulness. This approach not only enhances our grasp of information presentation but also paves the way for the development of more effective techniques in spotting fake or misleading news.\n\nRecent studies have delved into this approach, highlighting the importance of analyzing language style, tone, and structure in identifying deceptive content. Style and language are features that have proven valuable in distinguishing between fake and true articles, and specific linguistic features have proven valuable in indicating potential biases or misrepresentations in online content. These studies underscore the emerging significance of leveraging linguistic analysis to discern trustworthy news in the digital age.\n\nIn this context, we propose harnessing the “5W1H” technique commonly employed by journalists to clearly present the key information of a news item in an explicit way. This method focuses on identifying the What, Who, Why, When, Where, and How elements within a text. By applying this technique, we can systematically evaluate the reliability of the language across these dimensions. Analyzing the presence of these fundamental journalistic questions offers a structured approach to gauge the linguistic integrity and potential biases within the content. Moreover, our challenge will utilize texts in Spanish, aiming to advance techniques of this nature specifically tailored for this language. This integration of journalistic methodology with linguistic analysis not only provides a comprehensive framework but also could pave the way for enhancing the authenticity and trustworthiness of information in the Spanish digital media landscape.",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    },
    {
      "name": "Submission File",
      "content": "You have to create a submission file where each row represents an example. This file must be CSV file with the comma separator. Below we show examples for this subtask:\n\nId,Reliability_Label\n0,confiable\n1,confiable\n2,no confiable\n3,semiconfiable",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    }
  ]
}