# Efficiency 3rd Place : Public Model + Simple Post Processing

I'd like to thank Kaggle, the organisers and all the winners of this very interesting challenge. I want also to
SPECIALLY THANK
[@medali1992](https://www.kaggle.com/medali1992)
. Without him, I could'nt have made it to this 3rd place. Indeed, I used his public model shared
[here](https://www.kaggle.com/datasets/medali1992/mod-gte-base-weights)
.
This thread is intented to share the approach I used to reach efficiency 3rd place.
1- Main Idea
The main idea was to adjust the similarity matrix derived from model embeddings. The adjustment is based on couple (
ConstructName
,
MisconceptionName
). The approach I used, was quite similar to the one used in the beginning of the competition. We create a prompt by concatenating
SubjectName
,
ConstructName
,
QuestionText
and
AnswerText
. Once we get the embedding of the prompts and the 2587 misconceptions, We use cosine similarity to predict the top 25.
But I noticed that, they were
blattant inconsistencies
in the top 25 that can be sorted by
precomputed pointers
.  For instance :
1) for question related to pie charts we can get misconceptions related to simultaneous equations in the top 25.
2) for question related to gradient we can get misconceptions related triangles in the top 25.
Construction & Post Processing rule
That's the reason why, I decided to build pointers for the couples (
ConstructName
,
MisconceptionName
). I split the misconceptions in two groups. The first group \(M_{-}\) contains all the 1623 misconceptions present in the training dataset, the second group \(M_{+}\) the 964 remaining misconceptions. Now for a
ConstructName
\(c\) present in the training dataset, we associate two groups of misconceptions. The first group \(S_c\) contains all the misconceptions that can safely be associated with \(c\), and the second group \(P_c\) all the misconceptions that can potentially be associated with \(c\).  For a new misconception \(c\) not present in the training dataset, we decided to take \(S_c=\emptyset\)  and \(P_c\ = M_{+}\). It means that for a new misconception, we have no guarantee of a safe association, and we take all the unused misconceptions as potential assocation (to be more inclusive).
The adjustment rule used two parameters \(\alpha\) and \(\beta\).  For a sample with  prompt embedding \(x\),
ConstructName
\(c\) and a misconception \(m\) with embedding \(x_m\); We adjust the similarity as follow:
\(  S(c,m) = x^T.x_m + \alpha  \) if \(m \in S_c \)
\(  S(c,m) = x^T.x_m + \beta  \) if \(m \in P_c \)
\(  S(c,m) = x^T.x_m   \)  else.
Limitations
They are
two drawbacks
for this post-processing :
First  for the new
ConstructName
, we only adjust the
unseen
MisconceptionName
. By doing this, we are assuming that the most probable misconceptions are the unseen ones. So in the case where the true misconception is in the training dataset, it is less likely for us to get it in the top 25.
The choice of \(\alpha\) and \(\beta\) : For instance if  \(\alpha - \beta\) is too big, we can constrain the top 25 to almost the same misconceptions. At least, these parameters shoud be chosen by cross-validation. Yet I didn't have time to do it, since I was very active in this challenge only the first month.
2- Notebooks
There is nothing special here. Since this approach was intended for the efficiency track, we first pre-computed the misconceptions embeddings in a prior notebook (to save time) and then use them in the very inference notebook as well as adjustment pointers.
The inference notebook can be found
[here](https://www.kaggle.com/code/ulrich07/eedi-cpu-starter/notebook)
.
The precomputed embeddings can be found
[here](https://www.kaggle.com/code/ulrich07/eedi-starter-sent-embs-cpu)
.
The similarity adjustment pointers can be found
[here](https://www.kaggle.com/datasets/ulrich07/eedi-handy)
in the picke file
dus.pkl
.
3- Potential Improvements
Of course, this work can be improved in diverse ways. But the one that came in my mind, was to use my pre-computed pointers to learn new embeddings both for
ConstructName
and
MisconceptionName
. In this way, we could effectively better handle the new
ConstructName
not adressed in the pointer file.