# #60: Data augmentation explained: Generating more stars

I'm surprised that data augmentation hasn't been used more widely in this competition. Here's my approach.
Background: Dim stars make noisy pictures
If you've ever taken a picture with your mobile phone in low-light conditions, you know the effect: The camera adapts its shutter speed, but the picture will be noisier. In our dataset, the main feature of a star is its brightness, and the less bright a star is, the more noise we get in its picture: Star 1 is brighter than Star 0, and the unseen stars in the test set are less bright than Star 0. Furthermore, the stars emit less energy at higher wavelengths, which means that the right part of the spectra is noisier than the left one.
The effect can be quantified: The variance of the signal is proportional to the signal itself (see
[Wikipedia: Shot noise](https://en.wikipedia.org/wiki/Shot_noise)
). For the depth predictions, the variance contributed by the shot noise is inversely proportional to the signal strength.
Implementation
If we want to create data for a new star which is less bright than the existing ones, we can multiply all the data by a scaling factor < 1. This multiplication reduces the variance by the square of the scaling factor so that the new variance is too low, and we need to compensate by injecting some additional variance.
():
    
    old_brightness = data.mean(axis=).mean(axis=)
    scaling = new_brightness / old_brightness.reshape(-, , )
    new_data = data * scaling
    time_binning =  
    rng = np.random.default_rng()
    new_data += rng.normal(scale=np.sqrt(( - scaling) * new_data / time_binning),
                           size=new_data) 
     new_data
This implementation only works for downscaling but not for upscaling. Fortunately the stars in the test dataset are all less bright then Star 1 and we don't need to upscale the data.
Conclusion
Data augmentation has helped me tune the predictions for the sigma values.