We will use the Hamming loss function to evaluate the performance:
\[ \frac{1}{|Y|} \sum_{i=1}^{|N|} \sum_{j=1}^{|H_i|} XOR(y'_{ij}, y_{ij}) \]
where \\(  |N| \\) is the total number of images, \\( |Y| \\) is the total number of labels, \\(  |H_i| \\) is the number of labeled hashtags of image  \\( i \\).
The \\(  y'_{ij} \\) is a binary output (True or False ) that predicts the existing of a hashtag  \\( h_{ij}  \\). The $y_{ij}$ is the ground truth.