# Silver - LB 0.770 - Two Lines of Code!

Thanks Kaggle and Google for hosting a fun ASL competition.
I joined a few days ago, so I didn't have time to build my own model. Instead, I began with the best public notebook
[here](https://www.kaggle.com/code/saidineshpola/aslfr-ctc-based-on-prev-comp-1st-place?scriptVersionId=139048726)
(version 17) and attempted to improve it. After making a few changes, I boosted the LB from 0.700 to an amazing 0.770! and obtained Silver medal !!
Change Two Lines of Code
The best public notebook is Rohith Ingilela's awesome public notebook
[here](https://www.kaggle.com/code/irohith/aslfr-ctc-based-on-prev-comp-1st-place)
which was improved by Saidineshpola
[here](https://www.kaggle.com/code/saidineshpola/aslfr-ctc-based-on-prev-comp-1st-place?scriptVersionId=139048726)
. Version 17 of Saidineshpola's notebook achieves CV = 0.689 (scroll to bottom of version 17) and LB = 0.697.
That notebook uses TF Records made by Rohith Ingilela
[here](https://www.kaggle.com/code/irohith/aslfr-preprocess-dataset-tfrecords-mean-std)
. An easy trick to boost the performance of the public notebook is create TF Records which keep frames where hands are missing. The TF Records made by linked notebook removes all frames without hands. Instead we can use
"output two"
below and keep 50% of the frames with missing hands below to boost CV and LB. Updated notebook published
[here](https://www.kaggle.com/code/cdeotte/2-lines-of-code-change-lb-0-760)
.
![](assets/91c282a1143fb2544e832c30.png)
Here is preprocess code to use when making TF Records and during inference:
hand = tf.concat(, axis=)
hand = tf.where(tf.math.is, , hand)
mask = tf.math.not, )
alternating_tensor = tf.math.equal( tf.cumsum(
    tf.ones ))%,  )
mask = tf.math.logical
CTC (Connectionist Temporal Classification) Loss:
In this competition, I learned about CTC loss. This loss is amazing! It allows us to create a model with variable length input and predict variable length output in one step. So we can do things like seq2seq without waiting for a sequential decoder to decode each step. Instead we predict the entire output at once super fast! Giving the model some of the frames with hands missing helps identify duplicates and transistions:
![](assets/3f7a5ee5fcea4334f4bbdfb1.png)
Time Augmentation LB +0.004!
The public notebook trains for 50 epochs, I found that training for more epochs continues to boost CV and LB score! My final submission trains for 200 epochs. Furthermore, we can add augmentation (i.e. regularization) which helps the model train longer, prevent overfitting, and generalize better.
Below we see the histogram of train data Number of Frames divided by Character Length of Target Phrase. From this plot, we see that the ratio varies a lot. Some videos have a different recorded frame rate than others and some participants sign faster than others.
![](assets/12a0283d1f73566fba43718c.png)
What this means is that the model has a hard time transfer learning one frame rate to learn about another frame rate. We can help the model by using time augmentation. For each input sequence, we can randomly shrink frame length by 50% or enlarge 150%. This will add lots of new train data and help the model learn about different frame rates. Here is augmentation code:
tf.random.uniform(shape=(), minval=, maxval=)<.:
     = tf.math.round( tf.random.uniform(
        =(), minval = tf.cast(tf.shape(lip)[],tf.float32) / ., 
         = tf.cast(tf.shape(lip)[],tf.float32) * .) )
     x in
Post Processing LB +0.004!
Sometimes the model doesn't make a good prediction. The shortest train data is length 3. When the model predicts length 2 or less, we know it is a bad prediction. Therefore we can replace bad predictions with the best constant length prediction. Anokas found the best constant length prediction
[here](https://www.kaggle.com/code/anokas/static-greedy-baseline-0-157-lb)
. We add this to our TF Lite mode with the following code:
x = tf(tf(x) < , lambda: tf(
    , tf.int64), lambda: tf(x))
Solution Code
I published a Kaggle notebook
[here](https://www.kaggle.com/code/cdeotte/2-lines-of-code-change-lb-0-760)
demonstrating the above 3 changes. The first four bullet points below achieve
LB = 0.763
, then time augmentation boost
+0.004
and PP boost
+0.004
. Note that the second bullet point (about batch size) just makes things faster but doesn't change the CV nor LB score. The other bullet points are the key:
Change 2 lines of code to keep 50% missing hand frames
Change batch size from 32 to 128 and learning rate from 1e-3 to 4e-3
Change FRAME_LEN from 128 to 216
Increase epochs 50 to 200
Add time augmenation
Add post process for preds less than 3 chars