# Private 7th (Public 2nd) Place Solution Summary

First of all, I'd like to thank kaggle team, host for hosting this competition , and my team mates (@tereka
[@irrohas](https://www.kaggle.com/irrohas)
[@aerdem4](https://www.kaggle.com/aerdem4)
)
I finally got 5th gold medal. I'm really happy to be Competitions Grand Master.
Overview
Voting Ensemble of 3pipelines (Retriever + Reranker)
All reranker inference with vllm
Pipeline1 by Masaya
This is brief summary.
Masaya will write up detail on this
[link](https://www.kaggle.com/competitions/eedi-mining-misconceptions-in-mathematics/discussion/551430)
.
Retriever
Generate test answer with pretrained qwen35B
model :SFR-Embedding-2_R, Hard Negative Sampling(N=100) removed if unseen in training data.
dataset: Filled unknown Misconception with GPT4o-mini
candidate 60
Reranker
Qwen2.5 32B AWQ+LoRA (with 40 options)
options → 0~9 + A~Z + a~z (40 charactors)
dataset: only original
candidate 40
run twice and merge ([:40] and [20:])
left 25 with logprobs
Pipeline2 by Tereka+Kaeru
Retriever
model1 :SFR-Embedding-2_R、Hard Negative Sampling(100）
dataset: original + synthesis with gemma 27B/Qwen2.5 32B
lb 0.378
model2 : public model by sayoulala
weighted avg model1 and model2
candidate 70 and filter with masaya preds
only single fold
Reranker
Qwen2.5 32B AWQ+LoRA (with binary options)
options → yes, not
dataset: original + synthesis unknown misconceptions with gemma2 27B/Qwen2.5 32B
only single fold
Pipeline3 by Ahmet
Retriever
Omitted in final pipeline
Reranker
Candidates are each top9 from previous pipeline
Qwen2.5 32B (with 9 options)
Ensembling
Voting ensemble of each pipeline
Apply rank like following, join, sum rank, sort by sum_rank
sub0
  .with_columns(pl.col()..split())
  .with_columns(pl.lit([/i  i  ((, , ))]).alias())
  .explode(pl.col([, ]))
submission
  .sort(by=[, , , ], descending=[, , , ])
  .group_by([], maintain_order=)
  .agg(pl.col())
Postprocessing
Boost missing misconception rank
Accelerating Tips
Retreiver
Run parallel with cuda:0 and cuda:1
Pre-calculating Misconception Vector
Dynamic padding with collate_fn
Reranker
Inference with vllm (AWQ 4bit quantization)
Set vllm parameter enabling_prefix_cache True
Filter candidate with previous preds of pipeline
What not worked
Retrieval
Concat vectors
Avaraging many vectors
Reranker
Full-data trained model
QwQ model
Thank you!!