{
  "fetched_at": "2026-10-24T15:35:22Z",
  "source_url": "https://www.kaggle.com/competitions/edm-cup-2023/data",
  "pages": [
    {
      "name": "data-description",
      "content": "# \\*\\_unit\\_test\\_scores.csv\n\nThese files reference the unit test scores, which are the scores this competition revolves around predicting. training\\_unit\\_test\\_scores.csv contains all the unit test scores for you to use during model training, and evaluation\\_unit\\_test\\_scores.csv contains only the assignment log IDs for which you must predict the unit test scores.\n\n### assignment\\_log\\_id\n\nA unique identifier for a particular assignment completed by the student. Details on all the assignment logs in this data set can be found in assignment\\_details.csv file. You will notice that for assignment log IDs in evaluation\\_unit\\_test\\_scores.csv, the start and end time of the assignment is removed because that information is determined after the unit test is assigned and would provide unfair information about students' unit test performance. assignment\\_relationships.csv can be used to determine which assignments the student completed within the unit before taking their unit test.\n\n### problem\\_id\n\nA unique identifier for the problem within the assignment the student completed.\n\n### score\n\nThe score the student received on their problem. The score is 1 when the student completed an open-ended response problem or submitted the correct answer on their first attempt without any tutoring, and 0 otherwize. A list of every problem in each unit test can be found in sequence\\_details.csv, which should help with the score predictions.\n\n# \\*\\_relationships.csv\n\nThese files track which sequences and assignments are unit tests, and which sequences and assignments within each unit correspond to the unit tests. Each sequence ID corresponds to a distinct set of mathematics problems that a teacher can assign to their class, and each assignment log ID corresponds to a particular instance of a student starting a problem set assigned to them. These tables are very important because they inform which information from action\\_logs.csv corresponds to each unit test score that must be predicted.\n\n### unit\\_test\\_\\*\\_id\n\nThis is the ID of the unit test sequence or assignment log. The same ID will appear multiple times in this column; once for each sequence or assignment log within the unit.\n\n### in\\_unit\\_\\*\\_id\n\nThis is the ID of a sequence or assignment log with the unit that corresponds to the unit test. For sequences, there is a fixed set of in unit sequences for each unit. For assignment logs, each student could have been assigned different sequences from within each unit. Therefore, even if two unit test assignment logs correspond to the same sequence, they could have different sets of in unit assignment logs, because each unit test assignment log is related to only the assignment logs the student completed within the unit prior to taking the unit test.\n\n# action\\_logs.csv\n\nThis file contains all the clickstream data from students as they completed assignments within each unit. This file will likely be the main source of information for predicting unit test scores.\n\n### assignment\\_log\\_id\n\nA unique identifier for the assignment being completed by the student when they took an action.\n\n### timestamp\n\nThe UNIX timestamp for the instant the action was taken.\n\n### problem\\_id\n\nA unique identifier for the problem being completed by the student when they took an action. This field is only available when the student takes an action related to a problem.\n\n### max\\_attempts\n\nWithin ASSISTments, students potentially see a credit bar as they complete each problem. When students request tutoring or submit wrong answers, the credit bar decreases. This column indicates the maximum attempts a student has for a particular problem before they see they have lost all credit on the problem. This column's value, configurable by the teacher upon assignment, is only present for the problem\\_started action.\n\n### available\\_core\\_tutoring\n\nWithin ASSISTments, students have access to, at most, one core tutoring option when they start a problem, after this tutoring option has been exhausted, the student can request the answer to the problem. Alternatively, teachers can configure assignments to not provide any tutoring or answers to students, usually to mimic a testing environment. This column indicates which core tutoring option was available at the start of the problem. This column's value is only present for the problem\\_started action.\n\nOn occasion, students could also request auxiliary tutoring in the form of chat sessions with live tutors or skill-related videos. While the live\\_tutor\\_requested and skill\\_related\\_video\\_requested actions correspond to this auxiliary tutoring, the use of auxiliary tutoring was rare, and information on its availability at the beginning of a problem was not available.\n\nThere are four possible values for the available\\_core\\_tutoring column:\n\n- **hint** : A set of hints was available upon the student's request.\n- **explanation** : An explanation was available upon the student's request.\n- **answer** : Only the correct answer was available upon the student's request.\n- **no\\_tutoring** : No tutoring was available for this problem at all.\n\n### score\\_viewable\n\nA binary indication of whether or not the student can view the credit bar for the problem they are completing. Teachers configure whether or not their students can see the credit bar upon assignment. This column's value is only present for the problem\\_started action.\n\n### continuous\\_score\\_viewable\n\nA binary indication of whether or not the credit bar for the problem being completed shows partial credit or simply goes to zero upon the student submitting a wrong answer or requesting tutoring. Teachers configure whether or not their students can see their partial credit on problems upon assignment. This column's value is only present for the problem\\_started action.\n\n### action\n\nAn indication of which action was taken by the student in this particular instance.\n\nThere are fourteen possible values for the action column:\n\n- **problem\\_started** : The student started a problem.\n- **problem\\_finished** : The student finished a problem.\n- **continue\\_selected** : The student pressed the continue button after they finished a problem, moving them to the next problem in their assignment.\n- **correct\\_response** : The student submitted a correct response to a problem.\n- **wrong\\_response** : The student submitted an incorrect response to a problem.\n- **open\\_response** : The student submitted a response to an open response question.\n- **assignment\\_started** : The student started their assignment.\n- **answer\\_requested** : The student requested the answer to a problem.\n- **assignment\\_finished** : The student completed their assignment.\n- **assignment\\_resumed** : The student resumed their assignment. Unfortunately, there is no way to know exactly when they left their assignment, meaning this action often comes after a long delay from the previous action.\n- **hint\\_requested** : The student requested one hint from a set of hints for the problem.\n- **explanation\\_requested** : The student requested an explanation for the problem\n- **skill\\_related\\_video\\_requested** : The student requested a video that provides general instruction for the skill most related to solving the problem.\n- **live\\_tutor\\_requested** : The student requested a chat session with a live tutor.\n\n### hint\\_id\n\nA unique ID for the hint requested by the student. This column's value is only present for the hint\\_requested action.\n\n### explanation\\_id\n\nA unique ID for the explanation requested by the student. This column's value is only present for the hint\\_requested action.\n\n# sequence\\_details.csv\n\nThis file contains at least one row for each sequence present in the dataset and contains all the information unique to each sequence. Some sequences have duplicate rows because they are present in multiple locations within units. Whether that is intentional is unclear.\n\n### sequence\\_id\n\nA unique identifier for the sequence.\n\n### sequence\\_folder\\_path\\_level\\_1\n\nSequences are organized into folders, this is the highest level folder path, which contains information on whether the sequence came from the Kendall Hunt Illustrative Mathematics curriculum ([https://im.kendallhunt.com/](https://im.kendallhunt.com/)) or the Engage New York mathematics curriculum ([http://www.nysed.gov/curriculum-instruction/engageny](http://www.nysed.gov/curriculum-instruction/engageny)). If there is no folder at this path depth, this column is blank.\n\n### sequence\\_folder\\_path\\_level\\_2\n\nSequences are organized into folders, this is the second highest level folder path, which contains information on what grade or subject the sequence is a part of. If there is no folder at this path depth, this column is blank.\n\n### sequence\\_folder\\_path\\_level\\_3\n\nSequences are organized into folders, this is the third highest level folder path, which contains information on what unit the sequence is a part of. If there is no folder at this path depth, this column is blank.\n\n### sequence\\_folder\\_path\\_level\\_4\n\nSequences are organized into folders, this is the fourth highest level folder path, which contains information on what specific subject within the unit the sequence is a part of. If there is no folder at this path depth, this column is blank.\n\n### sequence\\_folder\\_path\\_level\\_5\n\nSequences are organized into folders, this is the fifth highest level folder path, which contains information on what specific subject within the unit the sequence is a part of. If there is no folder at this path depth, this column is blank.\n\n### sequence\\_name\n\nThe name of the sequence, which conveys information on what material the sequence is evaluating and what role it has within the curriculum.\n\n### sequence\\_folder\\_position\n\nThis specifies the order that the sequence appears within the folder. The sequences are ordered within the folder to align with when the curriculum suggests that they be assigned.\n\n### sequence\\_problem\\_ids\n\nA list of the ID of every problem contained within the sequence.\n\n# assignment\\_details.csv\n\nThis file contains one row for every assignment started by a student in the scope of the dataset.\n\n### assignment\\_log\\_id\n\nA unique identifier for each assignment started by a student\n\n### teacher\\_id\n\nA unique identifier for the teacher of the student that started the assignment.\n\n### class\\_id\n\nA unique identifier for the class the student that started the assignment was in.\n\n### student\\_id\n\nA unique identifier for the student that started the assignment.\n\n### sequence\\_id\n\nA unique identifier for the sequence of problems that was assigned to the student.\n\n### assignment\\_release\\_date\n\nThe UNIX timestamp for when the assignment was made available to the student.\n\n### assignment\\_due\\_date\n\nThe UNIX timestamp for when the assignment was due. If a teacher did not specify a due date, this column will be blank.\n\n### assignment\\_start\\_time\n\nThe UNIX timestamp for when the student started the assignment. For assignments that you must predict the score of, this column's information is removed because it could give an unfair predictive advantage.\n\n### assignment\\_end\\_time\n\nThe UNIX timestamp for when the student completed the assignment. If the student never completed the assignment, this column will be blank. For assignments that you must predict the score of, this column's information is removed because it could give an unfair predictive advantage.\n\n# problem\\_details.csv\n\nThis file contains one row for every problem referenced in the dataset, except for some problems in the action logs, which have been deleted from the database. These problems likely had errors during their original transcription into ASSISTments that were corrected, but no record of the original problems was kept.\n\n### problem\\_id\n\nA unique identifier for each problem.\n\n### problem\\_multipart\\_id\n\nSome problems are part of a multi-part problem. For example, the first problem in a multi-part problem might say \"If a circle has a radius of 5, what is its circumference?\" and the second problem might say \"What is its area?\". The second problem requires the context provided by the first problem. This column contains a unique identifier for each set of problems in a multi-part problem. Every problem has a multi-part problem ID, but it could be that there are no other problems with the same multi-part ID, meaning the problem is a standalone problem.\n\n### problem\\_multipart\\_position\n\nAn indication of the problem's order within the multi-part problem. The problem that comes first is at position 1, and the following problems each increment their position by 1 from the previous problem.\n\n### problem\\_type\n\nThe format of the answer to the problem.\n\nThere are 10 possible values for the problem\\_type column:\n\n- **Multiple Choice** : The student selects one of multiple predefined answers.\n- **Ungraded Open Response** : The student writes in plain text and/or uploads media for the teacher to see.\n- **Number** : The student must type in a number with no letters or mathematical symbols for it to be accepted as a submission, otherwise the student is told their answer is not in the correct format, and to revise it before submitting.\n- **Algebraic Expression** : The student must type an expression that contains only numbers, letters (when they are used as variables in the expression), and mathematical symbols for it to be accepted as a submission, otherwise the student is told their answer is not in the correct format, and to revise it before submitting.\n- **Numeric Expression** : The student must type an expression that contains only numbers and mathematical symbols for it to be accepted as a submission, otherwise the student is told their answer is not in the correct format, and to revise it before submitting.\n- **Check All That Apply** : The student must select between none and all of a set of predefined answers.\n- **Exact Match (ignore case)**: Without limiting the characters the student is allowed to input, the student must type an answer that exactly matches the problem's correct answer with the exception of using upper or lower case letters.\n- **Exact Fraction** : The student must type an answer that is a fraction that uses the same numerator and denominator as the problem's correct answer for the student to be considered correct, unlike a Numeric Expression problem type, which would be considered correct as long as the student's fraction simplified to the same thing as the problem's correct answer.\n- **Exact Match (case sensitive)**: Without limiting the characters the student is allowed to input, the student must type an answer that exactly matches the problem's correct answer.\n- **Ordering** : The student must arrange a set of predefined answers to place them in the correct order.\n\n### problem\\_skill\\_code\n\nThe Common Core State Standards for Mathematics skill code most related to solving the problem. If a skill code has a dash in it, that specifies a more detailed sub-skill beyond what the common core provides. The common core skill codes can be found here: [https://learning.ccsso.org/wp-content/uploads/2022/11/ADA-Compliant-Math-Standards.pdf](https://learning.ccsso.org/wp-content/uploads/2022/11/ADA-Compliant-Math-Standards.pdf).\n\n### problem\\_skill\\_description\n\nA brief description of the skill code the problem is tagged with.\n\n### problem\\_contains\\_image\n\nA binary indicator of whether or not the problem contains an image.\n\n### problem\\_contains\\_equation\n\nA binary indicator of whether or not the problem contains a Wiris Editor equation. More information on the Wiris editor can be found here: [https://demo.wiris.com/mathtype/en/](https://demo.wiris.com/mathtype/en/).\n\n### problem\\_contains\\_video\n\nA binary indicator of whether or not the problem contains a video.\n\n### problem\\_text\\_bert\\_pca\n\nThis is the first 32 principal components of the bert embedding for the text-based content of the problem across all the problem's bert embeddings, rounded to the 8th decimal place.\n\n# hint\\_details.csv\n\nThis file contains one row for every hint referenced in the dataset, except for some hints in the action logs, which have been deleted from the database. At one point, there were some hints in ASSISTments that simply gave away the answer with no elaboration. The hints missing from this file were likely removed for this reason and replaced with better tutoring.\n\n### hint\\_id\n\nA unique identifier for each hint.\n\n### hint\\_creator\\_id\n\nA unique identifier for the creator of the hint.\n\n### hint\\_position\n\nWhen hints are provided as tutoring for a problem, they come as a set of hints, where each hint reveals slightly more about how to solve the problem. This column indicates the order that the hint appears in the set of hints for its problem. The hint that comes first is at position 1, and the following hints each increment their position by 1 from the previous hint.\n\n### hint\\_contains\\_image\n\nA binary indicator of whether or not the hint contains an image.\n\n### hint\\_contains\\_equation\n\nA binary indicator of whether or not the hint contains a Wiris Editor equation. More information on the Wiris editor can be found here: [https://demo.wiris.com/mathtype/en/](https://demo.wiris.com/mathtype/en/).\n\n### hint\\_contains\\_video\n\nA binary indicator of whether or not the hint contains a video.\n\n### hint\\_text\\_bert\\_pca\n\nThis is the first 32 principal components of the bert embedding for the text-based content of the hint across all the hint's bert embeddings, rounded to the 8th decimal place.\n\n# explanation\\_details.csv\n\nThis file contains one row for each explanation in the dataset.\n\n### explanation\\_id\n\nA unique identifier for each explanation.\n\n### explanation\\_creator\\_id\n\nA unique identifier for the creator of the explanation.\n\n### explanation\\_contains\\_image\n\nA binary indicator of whether or not the explanation contains an image.\n\n### explanation\\_contains\\_equation\n\nA binary indicator of whether or not the explanation contains a Wiris Editor equation. More information on the Wiris editor can be found here: [https://demo.wiris.com/mathtype/en/](https://demo.wiris.com/mathtype/en/).\n\n### explanation\\_contains\\_video\n\nA binary indicator of whether or not the explanation contains a video.\n\n### explanation\\_text\\_bert\\_pca\n\nThis is the first 32 principal components of the bert embedding for the text-based content of the explanation across all the explanation's bert embeddings, rounded to the 8th decimal place.",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    },
    {
      "name": "Description",
      "content": "**Now that the competition has ended, all the data and instructions on how to submit an article to the JEDM special issue can be found here: [https://osf.io/yrwuh/](https://osf.io/yrwuh/)**\n\nUnderstanding how students’ behaviors affect their long-term performance has a wide range of applications. Knowledge tracing, item response theory, affect detection, dropout prediction, gaming detection, and many other common methods explored in educational data mining stem from the idea that the actions students take while they complete their assignments can be used to model latent variables that predict learning and performance.\n\nIn this competition, you are tasked with predicting students’ scores on end-of-unit assignment problems given their click-stream data across all the in-unit assignments they completed previously. This task will rely heavily on the methods used to extract relevant features from millions of actions taken by students within the ASSISTments online learning platform as they completed their mathematics assignments. Additionally, this data set includes information on the curricula, assignments, problems, and tutoring provided to the students to help inform your predictions.\n\nParticipants in this competition are invited to submit to a special issue of [The Journal of Educational Data Mining](https://jedm.educationaldatamining.org/index.php/JEDM). Papers should focus on the approach used in this competition to understand and model students' behavior, with emphasis on interpretable insights and how these insights can help individual students and improve online learning in general. Acceptance to the special issue will be based on a combination of paper quality and leaderboard rank.\n\nSubmissions to the special issue are managed through JEDM and can be submitted here: [https://jedm.educationaldatamining.org/index.php/JEDM](https://jedm.educationaldatamining.org/index.php/JEDM) through the **Special Issue EDM Cup 2023** Track.\n\nInstructions on how to submit can be found here: [https://osf.io/yrwuh/](https://osf.io/yrwuh/)\n\nThe competition ends May 31st, 2023 23:59:00 UTC.\n\nSubmissions to the special issue are due by the start of EDM 2023 (July 11th, 2023).",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    },
    {
      "name": "Evaluation",
      "content": "In this competition, you are predicting a binary measure of problem correctness. Therefore, the evaluation metric for this competition is [AUC](https://en.wikipedia.org/wiki/Receiver_operating_characteristic#Area_under_the_curve).\n\nThe submission file should contain two columns: **id** and **score**. **evaluation_unit_test_scores.csv** contains the corresponding information for each **id** that must be included in the submission file. **evaluation_unit_test_scores.csv** contains an empty **score** column; this should be filled in with predictions. **training_unit_test_scores.csv** contains the same information as **evaluation_unit_test_scores.csv** with the **score** column populated with problem scores for training purposes.\n\n",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    },
    {
      "name": "rules",
      "content": "By accessing this data, you agree to not attempt to identify ASSISTments users directly, or to modify or create any supplemental or third party datasets that identify ASSISTments users. If you are able to identify ASSISTments users in this or any ASSISTments data set, you will:\n- Email research@assistments.org with any supplemental files and a written explanation of how the data was de-anonymized\n- Delete all de-anonymized data and related outcomes\n- Halt pursuit of or redact publication of any outcome that risks identifying and/or harming ASSISTments users\n<br>\n\nFor a submission to be eligible for publication in the JEDM special issue, it must:\n- Attempt to model latent features of students' behavior (the model has to use the clickstream data in some way)\n- Provide a method to interpret the findings of the model (explain latent features, cluster behaviors, etc.)",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    }
  ]
}