The evaluation metric for this competition is
Mean Average Precision @ 10
Suppose there are m missing outbound edges from a user in a social graph, and you can predict up to 10 other nodes that the user is likely to follow. Then, by adapting the definition of average precision in IR (
[http://en.wikipedia.org/wiki/Information_retrieval](http://en.wikipedia.org/wiki/Information_retrieval)
,
[http://sas.uwaterloo.ca/stats_navigation/techreports/04WorkingPapers/2004-09.pdf](http://sas.uwaterloo.ca/stats_navigation/techreports/04WorkingPapers/2004-09.pdf)
), the average precision at n for this user is
ap@n = Σ
k=1,...,n
P(k) / min(m, 10)
where if the denominator is zero, the result is set zero; P(k) means the precision at cut-off k in the item list, i.e., the ratio of number of users followed up to the position k over the number k, and P(k) equals 0 when k -th item is not followed upon recommendation; n = 10
For example,
(1)     If the user follows recommended nodes #1 and #3 along with another node that wasn't recommend, then ap@10 = (1/1 + 2/3)/3 ≈ 0.56
(2)     If the user follows recommended nodes #1 and #2 along with another node that wasn't recommend, then ap@10 = (1/1 + 2/2)/3 ≈ 0.67
(3)     If the user follows recommended nodes #1 and #3 and has no other missing nodes, then ap@10 = (1/1 + 2/3)/2 ≈ 0.83
The mean average precision for N users at position n is the average of the average precision of each user, i.e.,
MAP@n = Σ
i=1,...,N
ap@n
i
/ N