Predictions are evaluated by the [Kendall tau correlation](https://en.wikipedia.org/wiki/Kendall_rank_correlation_coefficient) between predicted cell orders and ground truth cell orders accumulated across the entire collection of test set notebooks.

Let \\(S\\) be the number of swaps of adjacent entries needed to sort the predicted cell order into the ground truth cell order. In the worst case, a predicted order for a notebook with \\(n\\) cells will need \\(\frac{1}{2}n (n - 1)\\) swaps to sort.

We sum the number of swaps from your predicted cell order across the entire collection of test set notebooks, and similarly with the worst-case number of swaps. We then compute the Kendall tau correlation as:
\\[K = 1 - 4 \frac{\sum\_i S\_{i}}{\sum\_i n\_i(n\_i - 1)}\\]

You may find a Python implementation in this notebook: [Competition Metric - Kendall Tau Correlation](https://www.kaggle.com/ryanholbrook/competition-metric-kendall-tau-correlation).

## Submission File

For each `id` in the test set (representing a notebook), you must predict `cell_order`, the correct ordering of its cells in terms of the cell ids. The file should contain a header and have the following format:

```
id,cell_order
0009d135ece78d,ddfd239c c6cd22db 1372ae9b ...
0010483c12ba9b,54c7cab3 fe66203e 7844d5f8 ...
0010a919d60e4f,aafc3d23 80e077ec b190ebb4 ...
0028856e09c5b7,012c9d02 d22526d1 3ae7ece3 ...
etc.
```