# 10th place solution

First, I would like to thank the Armed Forces of Ukraine, Security Service of Ukraine, Defence Intelligence of Ukraine, State Emergency Service of Ukraine for providing safety and security to participate in this great competition, complete this work, and help science, technology, and business not stop and move forward.
TLDR: Transformer models with embedding based on Linear layers for pose,lips,eye, and STGCN + Linear layer for target hand
Preprocessing
Find target hand using nan in time dimension, flip if need, select poses where target hand is not nan, resize to 16 in time dimension, and finally normalize with
xyz = (xyz - np.nanmean(xyz[:,FACE,:],axis(,))) / np.nanstd(xyz[:,FACE,:],axis=(,))
Modeling
(All validation scores are based on
[@hengck23](https://www.kaggle.com/hengck23)
split on participant id)
After a stuck at 0.75 lb (~0.7 validation), I understand that I need to change strategy.I go back and start modelling only with target hand. After finding best normalization strategy using face I get ~0.69 val score with only hand.
Then I tried to add pose,lips,eyes I get ~0.71 val score. So for me was clear that most of the signal in dominant hand, so i tried to get different representation and ensemble it. Firstly I try to plot  target hand and then run 2.5d CNN or TransformerCNN based models, and get with this 0.6 val score only with hand. Then I tried to run  STGCN(Spatio-Temporal Graph Convolutional Networks) for target hand and get ~0.66 score. Because CNN gets too much time to process I decide to drop it. So finally after ensembling pose,lip,eye,hand,STGCN(hand) -> attention block i get ~0.72 score
x = torch.cat([
            self.pose_emb(x[:,:,:,:].view(B,L,-), x_mask)[],
            self.lip_emb(x[:,:,:+,:].view(B,L,-), x_mask)[],
            self.reye_emb(x[:, :, +       :++, :].view(B, L, -), x_mask)[],
            self.leye_emb(x[:, :, ++    :+++, :].view(B, L, -), x_mask)[],
            self.hand_emb(x[:, :, +++ :++++, :].view(B, L, -), x_mask)[],
            self.graph_emb(x[:, :,+++:++++, :].permute(, , , ).unsqueeze(-)),
            self.pos_embed[:L, :].unsqueeze().repeat(B,,)],axis=-)
Example how i ensemble it.
Then I started to add different augmentations (Here i have to spend definitely more time to increase my score even more), Most of the boost i get from :
Mixup
np.random.random() < self.args.mixup:
                    indices = torch.randperm(batch[].size(), device=batch[].device, dtype=torch.long)
                    beta = np.random.beta(, )
                    batch[] = beta * batch[] + ( - beta) * batch[][indices]
                    batch = self.run_logits(batch)
                    batch[] = beta * self.args.loss(logits=batch[], labels=batch[]) + (
                                 - beta) * self.args.loss(logits=batch[],
                                                           labels=batch[][indices])
Copy paste
np.random.random() < self.args.cutout:
                indices_to_shuffle = torch.randperm(batch[].size(), device=batch[].device, dtype=torch.long)
                cutout_len = self.args.cutout_len
                 (cutout_len) == :
                    cutout_len = random.uniform(*cutout_len)
                sz = (batch[].size() * cutout_len)
                ind = np.random.randint(, batch[].size() - sz)
                indices_to_cut = torch.(ind, ind+sz, device=batch[].device, dtype=torch.long)
                beta =  - cutout_len
                batch[][:, indices_to_cut] = batch[][indices_to_shuffle][:,indices_to_cut]
                batch[][:, indices_to_cut] = batch[][indices_to_shuffle][:,indices_to_cut]

                batch = self.run_logits(batch)
                batch[] = beta * self.args.loss(logits=batch[], labels=batch[]) + (
                         - beta) * self.args.loss(logits=batch[],
                                                   labels=batch[][indices_to_shuffle])
After that i get final score ~0.735-0.74 val score and 0.77 lb single model, 0.79 lb ensemble
What didn't work :
CNN on dominant hand, didn't get a lot of boost from it.
Different losses based on embedding for words.
Funny idea: Train model with arcface label : (pariticipant_id+sign nearly 5230 labels for full train) , get improvement in val score ~0.02 for val split based on label, because in lb we have samples from train too, i tried to add it to ensemble, but dont get any significant boost so decided to drop it.
One more time want to say thanks for every one who support Ukraine in such a hard times.
Slava Ukraini.