This competition will be evaluated computing the AUC score - same as the original one. You can compute it using sklearn's function: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_auc_score.html You can take a look at the samplesubmission.csv file to check how the submission should look like. It should have a column with SKID_CURR (ID) and default (target).