{
  "fetched_at": "2026-10-19T23:42:16Z",
  "source_url": "https://www.kaggle.com/competitions/cse-stat-416-24-sp-assignment-5/data",
  "pages": [
    {
      "name": "data-description",
      "content": "# Dataset Description\n\nFor this assignment, we will be looking at edX (anonymized) student data to try to predict whether a student received a certification (completed) a course. The target column is the `certified` column that you see in our training dataset.\n\n## File Descriptions\n\nThis assignment uses two files\n* `edx_train.csv` - The file containing the labeled training data\n* `edx_test.csv` - The file containing the unlabeled test data that will be used to evaluate your submission\n\n## Using these data files in your notebook\n\nHow you will access these files depends on which environment you are using.\n\n### Colab\n\nIf you're working with Colab Jupyter notebooks, you can use the starter code linked from the overview page to download the files to your Colab instance.\n\n```py\nsave_file('https://courses.cs.washington.edu/courses/cse416/23sp/homeworks/hw5/edx_train.csv', 'edx_train.csv') save_file('https://courses.cs.washington.edu/courses/cse416/23sp/homeworks/hw5/edx_test.csv', 'edx_test.csv')\n```\n\nYou can then refer to these files directly as `edx_train.csv` and `edx_test.csv`.\n\n### Kaggle\n\nThe starter notebook shows you the files available in the Kaggle system. For example, to refer to the training data you could use the file path `/kaggle/input/csestat-416-23sp-assignment-5/edx_test.csv`.\n\n### Local\n\nIf you set this up, you will need to download the data from Kaggle and then refer to the files in your code as you downloaded them on your computer.\n\n## Data Inputs\n\nThe data inputs are described in more detail in the `edX_column_description.pdf` below. We highly recommend that you read through this to understand what the columns mean.\n\nThe `userId_DI` field is necessary for evaluating your results but is likely not useful for your model in any way. You are encouraged to exclude `userId_DI` from being a feature that the model uses, but you must keep the `userId_DI` field so that you can properly identify the examples for evaluation.\n\nNote that some categorical values can have the value `unk` or `nan` for unknown. In the numerical categories, like `YoB` which represents Year of Birth, `-1` is filled in for unknown values. Unknown values will be present in the test set as well so your model will need to be able to handle them.",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    },
    {
      "name": "Description",
      "content": "## Overview\n\nFor this assignment, we will be looking at edX online course data from Harvard and MIT to predict completion of an online course based on the activity of a student. We have provided a training dataset and test dataset that you will use to train and evaluate your model (described in detail in the Data tab).\n\nYou will get more practice doing machine learning in the \"real world\" in the assignment this week. For many ML tasks, you will be given a dataset and your goal is to find the model that will do best in the future by using that dataset to train the model. Instead of risking deploying a bad model into production, a great test to verify that the model will work in the future is to evaluate it based on its accuracy on data it has never seen before.\n\nFor this assignment, we will do just that! You will have access to a training set with labels and a test set without labels. Your task is to train a model that will do optimally in the future, which will be assessed by looking at your predictions on this test dataset you haven't seen the labels for.\n\nKaggle is a good medium for simulating what it is like to deploy models in the real world! It assesses your model on these unseen examples and gives you an indication of how you will do by reporting your accuracy on a portion of the test set. This is seen as the \"leaderboard\" when you submit. The leaderboard is only meant to give you an indication of how your model might do in the future, but the final evaluation will be on data that has never been seen before or used to calculate leaderboard positions. This means you should take the leaderboard scores with a grain of salt since the final evaluation will be on a different dataset.\n\nThe idea of being assessed on data you have never looked at before might seem a bit unnerving at first, but this is exactly the scenario that happens whenever an ML practitioner deploys a new model. Uncertainty is unavoidable in the world, so we need to use the ML best practice we have learned in the class so far as to assure us that our model will do well in the future.\n\nAs a note, this assignment is NOT graded as a competition. The Kaggle leaderboard is just supposed to be a fun way for you to compare your model against your colleagues and the rankings are not used in grading. All that matters is how your model does in the end, not how it competes with everyone else's (more on grading in the Evaluation section).\n\n## Usernames\n\nWhen you make a submission on Kaggle, by default it will use your username on the leaderboard. If you would like to use something else, you may change your \"team name\" in the Team tab which will change what shows up in the leaderboard.\n\nRegardless if you change the name or not, you must include your username and/or team name in programming assignment 5 on Gradescope so we can identify which submission is yours. User/team names must not be offensive and should be appropriate. If we are not able to find which submission on Kaggle is yours, you will not receive credit for what you submit on Kaggle.\n\n## Group Work\n\nYou will be allowed to form teams for this assignment! This assignment should be doable alone, but we thought this was a good opportunity to let you explore working on machine learning in a team if that interests you. We recommend grouping up with your peers since you will learn a lot more from the experience!\n\nYou can form teams of up to 3 students. Kaggle lets you make submissions as a team and Gradescope allows you to turn in your code/report as a group as well. See Gradescope for more.\n\nWhen working with a group, you can fully collaborate on the programming portion (the report and the Kaggle competition), but you should do the concept portions individually like normal.\n\n## Starter Code\n\nThere is minimal starter code for this assignment [here](https://colab.research.google.com/drive/1_VcpxCBmOzqrHcqdCoKNuovcYqUh7F75?usp=sharing). Note that you will need to write everything based on what we have done in previous assignments.\n\nIt is recommended to work off of this starter notebook as it has section headings and instructions for all necessary components of the programming report.\n\n## Programming Report Requirements\n\nThe programming portion of this assignment involves writing code to train a model that accurately predicts whether students received their certification in the course for future student data examples of the same format.\n\nYou also need to turn in your notebook code on Gradescope (it could be several notebooks). Make sure they explain the process of what you are doing (use markdown cells), together with the prediction submissions file. Your notebook should clearly demonstrate the components listed below, and you should explain what you are doing and the interpretation of your results in Markdown (or Text) cells in your Jupyter notebook. The starter code has section headings for each key component.\n\nHere are several components we expect you to complete in your homework and should be described in markdown cells along with the code you used to complete them:\n\n* Your team name as it appears here on Kaggle.\n* Run at least 2 different types of classifiers (different methods: you can additionally run different versions of individual methods with different parameters and features) for prediction\n* Make a visualization that lets you compare the models you trained. Briefly discuss what you are inferring from this plot.\n* Discuss what features you used and whether you did some transformations on them. What features seemed important for your final model?\n* Explain what did you do to improve your initial predictions. Did it help?\n* Consider and discuss the ethical implications of using the model you trained. Imagine you were hired to work as a data scientist for an online education platform (exciting, right?). They want to use your model to predict which students they should tailor their course material towards so they can maximize their profits. Their idea is to use the model to help predict which student groups (i.e. from specific countries or certain educational backgrounds) would make them the most money by getting the most participants to complete their paid certificate program. Are there any ethical considerations we should think about before acting on that plan?\n\nThe starter notebook includes section headings for each of the instructions above.\n\n## Submitting\n\nFor this assignment, you will use Kaggle to turn in your model predictions, but everything else (Notebook, programming, concept questions) should be submitted on Gradescope. This will differ from regular assignments since we will not submit code on EdStem.\n\n## Logistics\n\nPlease do not use the Discussion tab on Kaggle, but instead, use EdStem so there is only one place to look. If you use Kaggle's Kernels to develop your code, please make sure they are private so that you are not in violation of our academic misconduct policy.\n\n## Coding Environment\n\nWe recommend getting used to a coding environment outside of EdStem for this environment. We recommend three possible options:\n\n1. Use Google Collaboratory (or Colab for short). Colab is Google's \"Google Docs for Jupyter Notebooks\". It works quite well and is very popular. You can use this starter notebook [here](https://colab.research.google.com/drive/1_VcpxCBmOzqrHcqdCoKNuovcYqUh7F75?usp=sharing).\n2. Use a \"kernel\" on Kaggle (click the Code tab above). This will be another Jupyter Notebook view that is built by the Kaggle team. This is another great option since it's integrated in Kaggle, but make sure you don't share your notebook publicly as that would be a violation of our academic honesty policy.\n3. Download Python and set up Jupyter Notebook on your computer locally. We recommend installing Python with [Anaconda](https://www.anaconda.com/download) and launching Jupyter Notebooks from there.\n\n## Grading\n\nFor information about how your solution will be evaluated and graded, see the Evaluation section.\n\n## Due Date\n\nThis Kaggle assignment for HW5 is due on **Thursday 5/16 11:59PM PST**. If you decide to use late days (up to two), you can submit up till Saturday 5/18 11:59PM PST. However, this Kaggle competition will be open until the end of the month, in case you want to play around with it more even after the homework is due.",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    },
    {
      "name": "Evaluation",
      "content": "## Overview\n\nGrading will be explained below, but we want to make clear that even though Kaggle presents things as a leaderboard, you are NOT being graded competitively. The leaderboard is supposed to be a fun way to see how your models compare to others and is not meant to imply that you are being graded based on your performance relative to others; as mentioned below, your grade is based on the accuracy of your model.\n\nThe evaluation metric used for this assignment is classification accuracy:\n\n$$\\text{Accuracy}(\\hat{f}, (x_i,y_i)_{i=1}^n) = \\frac{1}{n} \\sum_i^n \\text{IsCorrect}(i)$$\n\n$$\\text{IsCorrect}(i) = 1 \\text{ if } (y_i = \\hat{f}(x_i)) \\text{, } 0 \\text{ otherwise}$$\n\n## Submit to Kaggle\n\nFor your submission to Kaggle, you only need to submit a CSV file that has the Id and Category for each row in the test dataset. It will use the `userid_DI` and `certified` columns to compare to the true answers in order to evaluate your model.\n\n| `userid_DI` | `certified` |\n| --- | --- |\n| 123 | 1 |\n| 45 | 0 |\n| 678 | 0 |\n| ... | ... |\n\nTo simplify getting the Kaggle format right, we have provided a code snippet that you should use to save the Kaggle submission. This code is already included in the starter code.\n\n```py\ndf_test = pd.read_csv('edx_test.csv')\n\n# Any code to pre-process the data \n...\n\n# The code to make the predictions on the test data \n# (likely something like model.predict(...))\npredictions = ...\n\nto_save = df_test[['userid_DI']].copy()\nto_save.loc[:, 'certified'] = predictions\nto_save.to_csv('submission.csv', index=False)\n```\n\nYou can then upload `submission.csv` to Kaggle to evaluate the model after running this code.\n\n**Important Note:** Kaggle only lets you submit 10 times a day, resetting at 00:00 UTC. This will keep the test accuracies as correct as possible. We recommend you start early so you don't run up against the submission limit close to the deadline.\n\n**Important Note:** Kaggle uses the Id field for each example for evaluation. It's very important that your code that pre-processes and predicts the test data does not change the order that the examples appear in the test dataset. The code snippet above assumes the predictions variable stores a series of length `len(dftest)` that is in the same order as the original `dftest`. If you were to change the order of the dataset or predictions, you will likely get very wild accuracies on Kaggle.\n\nIf you want to test out if you can submit everything correctly, you can try a baseline classifier that selects labels at random. Such a classifier is shown as an example in the starter code.\n\n## Kaggle Leaderboards\n\nKaggle has the notion of a public leaderboard and a private leaderboard both based on portions of the test dataset provided.\n\nThe public leaderboard is viewable while the assignment is open. The score in the public leaderboard is based on the accuracy on 1/3 of the test data. The test accuracy of your model for this 1/3 of the test data is shown to you each time you submit.\n\nOnce the assignment has closed, the rankings are computed based on the accuracy on the remaining 2/3 of the test data; this is called the private leaderboard. This is similar to \"shipping\" your model and seeing how it performs in the wild on unseen data. This is the accuracy score that will be used for grading.\n\n## Grading\n\nThe majority of your grade for this assignment will be determined by your accuracy on the private test data, but it also includes a portion for your written report. Your grade breakdown for this assignment will be:\n\n* 40 points: Accuracy on the private test data\n* 20 points: Programming portion and written report\n\n(Recall there is also a Conceptual portion of HW5 on Gradescope, as usual. This Kaggle assignment is the Programming portion.)\n\nYou do not need to get 100% accuracy on the test data for full credit. We will give full credit for this portion of the assignment for any test accuracy on the private dataset that is 95% or above. Don't stress over the numbers, focus on explaining your strategy on Gradescope and polishing your notebooks.\n\nHere are some examples for how grades can work for this assignment, ignoring the concept portion:\n\n* 75% accuracy on private test data is an 75/95 which will be converted to 31.58 / 40, 15/20 on the programming report => 46.58/60 total\n* 95% accuracy on private test data is a 95/95 which will be converted to 40/40, 20/20 on the programming report => 60/60 total\n* 98% accuracy on private test data is a 95/95 which will be scaled to 40/40, 15/20 on the programming report => 55/60 total\n\n## Extra Credit\n\nAs a small incentive to encourage you to make the best models you can, we will provide a small amount of extra credit to the top 3 teams on the private leaderboard. Like other extra credit, this is not going to be something that will have a huge impact on your grade and is really meant to be something for fun and not to stress you out. Please ensure you are familiar with the extra credit policy on the syllabus.",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    },
    {
      "name": "rules",
      "content": "Make sure you read all rules and logistic info before starting the assignment.\n\nThis assignment falls under the Academic Honesty policies laid out in the [CSE/STAT 416 Syllabus](https://courses.cs.washington.edu/courses/cse416/24sp/syllabus).",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    },
    {
      "name": "abstract",
      "content": "**Make sure you read the following sections in their entirety before you begin.** There is important information about starter code, programming report requirements, Kaggle submission format, and more!\n\nThis is intended to be a fun assignment, so don't forget to have fun!",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    }
  ]
}