The evaluation metric for this competition is
[AUC](https://www.kaggle.com/wiki/AUC)
. The area under the Receiver Operating Characteristic (ROC) curve,measures accuracy using the statistics True Positive Ratio (TPR) and False Positive Ratio (FPR). TPR is the ratio of true positives (tp) to all samples (tp + fp + tn + fn). FPR is the ratio of false positives to all samples. The AUC is given by:
AUC(f) = \frac{\sum_{t_0 \in D^0}\sum_{t_1 \in D^1}1[f(t_0) < f(t_1)}{|D^0| \cdot |D^1|}]
where 1|f(t_0) < f(t_1)| denotes an indicator function which returns 1 iff f(t_0) < f(t_1) otherwise return 0; D^0 is the set of negative examples, and D^1 is the set of positive examples.
Before the submission deadline , you can only view the AUC evaluated on 50% of the dataset on the leaderboard. After the deadline, we will reveal your model AUC evaluated on the entire dataset to rate your performance. However, you do not need to worry that your score would be affected by 1 or 2 percent of AUC loss; we will mark based on the baselines your model defeats.
Submission Format
Your submission file should contain two columns: Id and Probability. Each test author node pair is allocated a unique id, and you need to use this id as your prediction index. For example, the first row
3280,50982
means a test node pair with author id 3280 and paper id 50982, its index is 0. If you predict there exists an edge between 3280 and 50982 with probability 0.86, the value of the corresponding row of your uploaded file is 0,0.86.
The file should contain a header and have the following format:
index,probability
0,0.86
1,0.21
Detailed description can be found in ‘data’ page.