Evaluation Metric
The evaluation metric for this competition is frame-level accuracy. There are a total of 169,656 frames in the test set. You will be ranked by unweighted accuracy on those phoneme state labels.
Submission Format
Submission files should contain two columns: Id and Label.
Id
: the 0-based index of the frame in the test set [0-169,655]
Label
: the predicted label of the phoneme [0-137]
Id=0 is the first frame in the first utterance. Id=169,656 is the last frame in the last utterance.
A sample submission file is available on the Data page.
Rules
Your model
must
use an MLP (no RNNs or ConvNets). We will check your code submitted to Autolab to make sure this is the case. However, you are free to use auto-differentiation libraries like PyTorch or Tensorflow.
Note: Support will only be provided for Pytorch.
Autolab
Important
: In order to receive a grade for this portion of the assignment, you must turn in all of the code you used to generate your Kaggle submission on Autolab. This means that
all of your code for HW1pt2 and a file called 'kaggle.txt' containing only your Kaggle user ID
should be put in a tarball (.tar file) and submitted here. Autolab will not be autograding your code, this is only to confirm that you wrote the code for the assignment yourself.
Grading
Submissions that outperform random classification are guaranteed a C grade.
Submissions that outperform a baseline provided by the course staff are guarenteed a B grade.
Top scoring (lowest error) submissions will receive grades in the A range.