# 20th place solution -- Keypoint-Based Dual-Graph Predictor Acknowledgement Although I'm actually solo (with [@daphne4sg](https://www.kaggle.com/daphne4sg) ) in this competition, I want to thank all my teammates [@goodcoder](https://www.kaggle.com/goodcoder) , [@bigochampion](https://www.kaggle.com/bigochampion) and [@cybersimar08](https://www.kaggle.com/cybersimar08) who help me in another LLM competition and share good things about personal life. I learn a lot from you guys. I also want to thank [@andrewjdarley](https://www.kaggle.com/andrewjdarley) for hosting such amazing competition. This project gives a lot of helps for my job interview. I have high willingness to do further development because your goal is very interesting. Short Story About This Solution This solution originates from the best project of my career, which I completed during my master’s degree a year ago. I collaborated with a director at Micron Technology through an industrial partnership. The project was extremely challenging—we were required to build a high-performance anomaly detector using only 88 HBM scanned images due to data privacy constraints. Additionally, the labels were a mix of AOI-judged results and human expert annotations. Our objective was to model the preferences of the human expert. Some parts of my BYU solution were adapted from this HBM project, such as keypoint generation and label inference from graph structures. However, the most innovative aspect of my approach was leveraging few-shot prediction with a vision-language model (VLM) instead of traditional object detectors like YOLO. Essentially, I used prompts such as “this HBM is anomalous” or “HBM is OK” to represent the presence or absence of a target, allowing the model to make an initial classification guess. My solution calibrates the text-image embedding space using graph-based methods to enable the VLM to perform effectively for our specific task. As a result, I was able to train the model with only about 20 images, and it successfully detected nearly all anomalies identified by human experts in the HBM dataset. Solution Initial Idea The initial idea can refer to the [notebook](https://www.kaggle.com/code/tom99763/first-idea-to-extract-keypoints-byu/notebook) where I made it public earlier. I want to produce a lot of points then refining them by building their relationship with labels. My strategy is sampling around the bacteria since motor existed in the head of it. If I aggregate all the keypoints and summarize them, the correct semantic might can be captured and get away from noisy labels. However, the biggest issue is how to represent the feature of each point, so I only use patch feature extracted from a simple CNN then building GNN to predict the location of motor. It turns out a really bad model. Further Improvement -- YOLO as Feature Extractor and Keypoint Generator It's funny that I almost leaked my solution in [this post](https://www.kaggle.com/competitions/byu-locating-bacterial-flagellar-motors-2025/discussion/573491) two month ago. I had short discussion with [@hengck23](https://www.kaggle.com/hengck23) about this development. After this discussion, I start to implement it and the idea becomes the pipeline showing below. So basically I use the C3K2 feature map of YOLO11 because the feature in each spatial position can represent refined object. Setting small confidence threshold (conf=0.05) can filter enough object locations determined from YOLO11. This design boosts the score, improving basic approach a lot. Node Feature -- C3K2 + RandomWalkPE Feature makes very big difference for motor prediction. I've tried almost every types of features like HoG, Daisy, EfficientNet, ResNet and different YOLO versions, all of them do not work well except YOLO11, but the score is still not good enough. After several days, I found AddRandomWalkPE on the [PYG document](https://pytorch-geometric.readthedocs.io/en/2.5.1/generated/torch_geometric.transforms.AddRandomWalkPE.html) . I suddenly realize this is the missing ingredient in 3d task. The score instantly boosts after I add it. So the node feature is C3K2 + RanadomWalkPE with walk_length=8 . I think the main reason it improves the score is because it can represent the position of each point, implicitly modeling the position of motor. Create More 3D points YOLO successfully generates a lot of points by setting low confidence threshold, but it is still sparse in 3d space and has high possibility missing object existence. So my strategy for solving this issue is using the predicted point as center then uniformly sampling other points within a radius. I set radius and num_samples to control the range and density, respectively. I found using this strategy for training GNN also boosts score. Label Development -- Similarity within Radius Since annotating a single point as motor can not completely represent the actual semantic (that's why host gives so much radius tolerance in the metric), I label a point as positive by the following conditions: Features similarity between current location and ground truth location larger than thr_sim . The current location is within the 3d radius thr of ground truth location. (): label = train_labels[train_labels.tomo_id == tomo_id] (, label[[, , ]].values[]) n_motors = label[].item() n_motors == : np.zeros(points.shape[], ) d, h, w = label[[, , ]].values[] loc = label[[, , ]].values[] tz, ty, tx = loc.astype() fd, fh, fw = feat.shape[], feat.shape[], feat.shape[] z = tz.astype() y = ((ty/h) * fh).astype() x = ((tx/w) * fw).astype() extract_feat = extract_feat / np.linalg.norm(extract_feat, axis=-, keepdims=) target_feat = feat[z, :, y, x][, :] target_feat = target_feat / np.linalg.norm(target_feat, axis=-, keepdims=) sim = extract_feat @ target_feat.T bool1 = sim[:, ]>thr_sim dist = np.linalg.norm(points - loc[, :], axis=-) bool2 = dist