## MiFID

Submissions are evaluated on MiFID (Memorization-informed Fréchet Inception Distance), which is a modification from [Fréchet Inception Distance (FID)](https://arxiv.org/abs/1706.08500). 

The smaller MiFID is, the better your generated images are.  



## What is FID?
Originally published [here](https://arxiv.org/abs/1706.08500) ([github](https://github.com/bioinf-jku/TTUR)), FID, along with [Inception Score (IS)](https://arxiv.org/abs/1606.03498), are both commonly used in recent publications as the standard for evaluation methods of GANs. 

In FID, we use the Inception network to extract features from an intermediate layer. Then we model the data distribution for these features using a multivariate Gaussian distribution with mean µ and covariance Σ. The FID between the real images \\(r\\) and generated images \\(g\\) is computed as:

\\[
\text{FID} = ||\mu\_r - \mu\_g||^2 + \text{Tr} (\Sigma\_r + \Sigma\_g - 2 (\Sigma\_r \Sigma\_g)^{1/2})
\\]

where \\(Tr\\) sums up all the diagonal elements. FID is calculated by computing the Fréchet distance between two Gaussians fitted to feature representations of the Inception network.



## What is MiFID (Memorization-informed FID)?
In addition to FID, Kaggle takes training sample memorization into account. 

The memorization distance is defined as the minimum cosine distance of all training samples in the feature space, averaged across all user generated image samples. This distance is thresholded, and it's assigned to 1.0 if the distance exceeds a pre-defined epsilon. 

In mathematical form:

\\[d\_{ij} = 1 - cos(f\_{gi}, f\_{rj}) = 1 - \frac{f\_{gi} \cdot f\_{rj}}{|f\_{gi}| |f\_{rj}|}\\] 

where \\(f\_g\\) and \\(f\_r\\) represent the generated/real images in feature space (defined in pre-trained networks); and \\(f\_{gi}\\) and \\(f\_{rj}\\) represent the \\(i^{th}\\) and \\(j^{th}\\) vectors of \\(f\_g\\) and \\(f\_r\\), respectively. 


\\[d = \frac{1}{N} \sum\_{i} \min\_j d\_{ij} \\]

defines the minimum distance of a certain generated image (\\(i\\)) across all real images ((\\(j\\)), then averaged across all the generated images. 

![enter image description here][1]
defines the threshold of the weight only applies when the (\\(d\\)) is below a certain empirically determined threshold. 

Finally, this memorization term is applied to the FID:
\\[ MiFID = FID \cdot \frac{1}{d\_{thr}}\\]

## Kaggle's workflow calculating MiFID for public and private scores
Kaggle calculates public MiFID scores with the pre-train neural network [Inception](http://download.tensorflow.org/models/image/imagenet/inception-2015-12-05.tgz), and the public images used for evaluation are the rest of the TFDS Monet paintings. **Note that as a Getting Started competition there is no private leaderboard.**

A demo of our MiFID evaluation code can be seen [here](https://www.kaggle.com/wendykan/demo-mifid-metric-for-dog-image-generation-comp).


## Submission File
You are going to generate 7,000-10,000 Monet-style images that are in `jpg` format. Their sizes should be 256x256x3 (RGB). Then you need to zip those images and your output from your Kernel should only have **ONE** output file named `images.zip`. 

Please note that Kaggle Kernels has a number of output files capped at 500. We highly encourage you to either directly write to a `zip` file as you generate images, or create a folder at `../tmp` as your temporary directory. 

> **Honor Code**: This competition has a unique format with images expected as submission, rather than predictions. Competitors should be using **generative methods** to create their submission images and not directly submit images of Monet paintings or altered versions of such images. This defeats purpose of a getting started competition aimed at learning.


  [1]: https://storage.googleapis.com/kaggle-media/competitions/GAN/latex-img-150.png
  [2]: https://storage.googleapis.com/kaggle-media/competitions/GAN/Kaggle%20GAN%20Diagram%20(3).png