The metric used in this challenge consists of 2 parts: Weighted Mean Absolute Error Ratio and Weighted Cosine Similarity. Importantly, WMAE is normalized against a baseline prediction (average of the training data). Submissions need to perform better than the baseline in terms of WMAE to get a positive score.

## Weighted Mean Absolute Error (WMAE)
Inspired by [Mejia et al.](https://arxiv.org/abs/2506.22641), we define the WMAE for a single perturbation as: 

$$
\mathrm{WMAE}(true, pred)
= \frac{1}{n}\sum_{i=1}^{n} w_i \,\left|\, true_i - pred_i \,\right|.
$$

where \\(true\\) and \\(pred\\) are the vectors of ground-truth and predicted delta expression values across \\(n\\) genes and \\(w = (w_1,\ldots,w_n) \in \mathbb{R}^n\\) is a vector of nonnegative weights with the same length as \\(true\\) and \\(pred\\), satisfying

$$
w_i \ge 0 \quad \text{for } i=1,\ldots,n,
\qquad \sum_{i=1}^{n} w_i = n.
$$

The weights vector is calculated using the moderated t-statistic from the [limma](https://www.bioconductor.org/packages/release/bioc/html/limma.html) package. 

Let \\(t = (t_1,\ldots,t_n) \in \mathbb{R}^n\\) be the vector of moderated t-statistics for a single perturbation. Define

$$
c_i = \min\bigl(|t_i|+0.1,\;10\bigr), \qquad i=1,\ldots,n.
$$

If the perturbation targets gene with index \\(g \in \{1,\ldots,n\}\\), set \\(c_g = 0\\).

Let 
$$
M = \max_{1\le j \le n} c_j
$$

We define the weight vector \\(w = (w_1,\ldots,w_n) \in \mathbb{R}^n\\) as

$$
w_i
= n \cdot
\frac{\left(\dfrac{c_i}{M}\right)^2}{\sum_{k=1}^{n}\left(\dfrac{c_k}{M}\right)^2},
\qquad i=1,\ldots,n.
$$


Finally, to aggregate over multiple perturbations, we define the WMAE ratio score \\(W\\) over \\(L\\) perturbations as

$$
W
= \sum_{l=1}^{L}
\min\left(
5,\;
\log_2 \left( \frac{WMAE^{base}_l}{WMAE^{pred}_l} \right)
\right).
$$

Here, \\(\mathrm{WMAE}^{\mathrm{base}}_l\\) is the WMAE for perturbation \\(l\\) of the baseline prediction, and \\(\mathrm{WMAE}^{\mathrm{pred}}_l\\) is the WMAE for perturbation \\(l\\) of the model prediction. The threshold of 5 serves only to safeguard against outlier cases. 

In this instance, the baseline prediction is the same for all perturbations and is defined as the arithmetic mean of the log fold-change vectors across the 80 training perturbations. Let \\(x^{(j)} \in \mathbb{R}^n\\) denote the log fold-change vector for training perturbation \\(j\\), for \\(j=1,\ldots,80\\). The baseline prediction vector is

$$
\bar{x} = \frac{1}{80}\sum_{j=1}^{80} x^{(j)}.
$$


## Weighted Cosine Similarity (Wcos)

We define the weighted cosine similarity score \\(Wcos\\) between two vectors \\(a,b \in \mathbb{R}^m\\). In our setting, \\(a\\) is the single concatenated ground-truth delta vector across all genes in all perturbations of length \\(m = L \times n \\), and \\(b\\) is the corresponding concatenated predictions vector.

Let the smoothstep function be defined as
$$
s(t) = t^2(3-2t).
$$

We fix the gating constants \\(left = 0\\) and \\(right = 0.3\\). For each coordinate in the concatenated vectors \\(i=1,\ldots,m\\), we define
$$
x_i = \max(|a_i|,\;|b_i|),
\qquad
t_i = \frac{x_i - left}{right - left} = \frac{x_i}{0.3},
\qquad
\tilde t_i = \min\bigl(1,\max(0,t_i)\bigr),
$$
and the (smooth) gate weight
$$
w_i = s(\tilde t_i).
$$

Let \\(w_i^2\\) denote the squared weights. Then the weighted cosine similarity is

$$
Wcos(a,b) = \frac{\sum\_{i=1}^{m} w\_i^{2}\, a\_i b\_i}{\left(\sum\_{i=1}^{m} w\_i^{2}\, a\_i^{2}\right)^{1/2}\left(\sum\_{i=1}^{m} w\_i^{2}\, b\_i^{2}\right)^{1/2}}.
$$

(If the denominator is zero, define \\(Wcos(a,b)=0\\).)

## The scoring metric
The final score used in this is the multiplication of the aggregated WMAE ratio scores and the weighted cosine.  

$$
W \times \max(0,Wcos)
$$

See [the implementation](https://www.kaggle.com/code/mylliabiotechnology/relative-wmae-multiplied-by-wcosine)