{ "fetched_at": "2026-10-20T00:16:09Z", "source_url": "https://www.kaggle.com/competitions/cs-4248-fact-checking-2420/data", "pages": [ { "name": "data-description", "content": "

Dataset Description

\n

The task is to do a mini-research for solving a fact-checking task. In this fact-checking task, you need to classify whether a text is an important fact (labeled as 1), an unimportant statement (labeled as 0), or a non-factual statement (labeled as -1).

\n\ntrain.csv contains 22,501 rows, where each row has three columns:\n\n\n

test.csv contains 1,032 rows, where each row has 2 columns, same as the train.csv but without the Verdict column.

", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Description", "content": "

Congrats! You found the second assignment

\n\n

Make sure you read the assignment pdf carefully. Rename your team name to your student number under the team page. We encourage you to use public tools/libraries for building your model. Afterward, you can read the data descriptions in the Data tab and the evaluation descriptions in the Evaluation tab.

\n\n

Integrity Note

\n**!! WARNING !!** All three assignments in CS4248, including this Assignment 2 (A2), are strictly individual assignments. These assignments are all crafted such that you, as an individual student, can accomplish the assignment on your own, without needing to consult others or AI tools. Explicitly, this means:\n\nInevitably, as AI-enhanced tools are being added to many software packages, you may find it difficult to avoid the implicit use of AI. Please ask us in the Assignment 2 discussion forum on Canvas -- public discussion on the forum is sanctioned, as long as you do not give away answers; if you are unsure, ask your tutorial leader, or us directly via email. \n\nWhile we enjoy having students talk and learn with each other, there are appropriate times to do this; but these assignments are not such an appropriate time. We have had to make the guidelines very explict (no sharing/consultation/discussion) since it is hard to demarcate what is acceptable and what is not. So we have had to forgo it for these assignments. \n\nFailure to follow these guidelines may result in prosecution that take months to resolve and may result in significant penalties to your academic career.\n\n

Limitations

\n
    \n
  1. Your implementation should be runnable in Python 3.11.x.
    2. \n
  2. To make it fair for everyone, the allowed machine learning algorithms are limited to the algorithms taught up to Week 6. Slight variations of the algorithm (different probability distribution for naive bayes, different activation functions), regularization, and normalization are allowed. RNNs are not allowed as it will be taught in Week 7.
    3. \n
  3. Feature engineering is highly encouraged. Finding good features and manually combining ones that you think need to be combined such that a linear classifier can benefit from it will be helpful. Using pre-trained models and standard libraries to generate feature is allowed.
    4. \n
  4. Using additional data instances (external data) is not allowed; only data augmentations of the provided data instances (e.g., upsampling) are allowed. This also means fine-tuning (continuing training or initializing weight) pre-trained models are not allowed.
    5. \n
\n\n

Submission Guidelines

\n

In order to make a submission to Kaggle, you need to :

\n
    \n
  1. Download train.csv and test.csv.
    2. \n
  2. Implement your code. You may use the skeleton code.
    3. \n
  3. Run the code. The skeleton code should automatically generate A2_YourName_E01234567.csv. (Do not forget to change to your student number in the code).
    4. \n
  4. Make sure A2_YourName_E01234567.csv contains 2 columns: Sentence_id and Verdict , and the number of rows match with the number of rows in test.csv (which has 1,032 lines). We encourage you to check the file since you are limited to submit up to 20 submissions per day.
    5. \n
  5. Log in to Kaggle, and head to Submit Predictions of this competition, then submit your A2_YourName_E01234567.csv
    6. \n
  6. See your score under the same page, or the leaderboard page.
    7. \n
  7. Please take note that the score shown in the leaderboard is only from 50% of the test data, as the rest will be evaluated privately. You may see your final score (private score) sometime after the Assignment 2 due date.
    8. \n
", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Evaluation", "content": "

Evaluation Metric

\n\n

The evaluation metric for this competition is Macro F1-Score. The F1 score, commonly used in information retrieval, measures accuracy using the statistics precision p and recall r. Precision is the ratio of true positives (tp) to all predicted positives (tp + fp). Recall is the ratio of true positives to all actual positives (tp + fn). The F1 score is given by:

\n

\\[ F1 = 2\\frac{p \\cdot r}{p+r}\\ \\ \\mathrm{where}\\ \\ p = \\frac{tp}{tp+fp},\\ \\ r = \\frac{tp}{tp+fn} \\]

\n

The F1 metric weights recall and precision equally, and a good retrieval algorithm will maximize both precision and recall simultaneously. Thus, moderately good performance on both will be favored over extremely good performance on one and poor performance on the other.

\n\n

Submission Format

\n

The submission files should contain two columns: Sentence_id and Verdict.\n

The file should contain a header and have the following format:

\n
Sentence_id,Verdict
1,1
2,0
", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "rules", "content": "This is a page where you can include rules that participants must accept before joining. You may wish to include rules like:\n - Don't cheat!\n - Apply yourself!\n - Have fun!", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false } ] }