# Contest methods

Surprisingly, one supervised method we tried on a whim to establish a supervised baseline ended up as the top performing model.  Here's a brief description of our method:
Though we explored various unsupervised and semi-supervised options, our best submission consisted of purely supervised features: the posterior probabilities output from Breiman's Random Forest algorithm.  Input features to this algorithm came from the
 union of two sets: the top k features with the most non-zeros, and the top k features with the largest difference between class means.  The sum of all features for each data point was included as well.  Only the labeled dataset was used to select these features
 and train the model.
A total of 5 submissions were included as final features, each comprising a different number of the top features from the two feature selection methods (ranging from ~600 to ~1400 total features) and slightly different random forest parameters.
This method got first on the public and private sets, with a private AUC of 0.9771.
Here's several of the alternatives we explored:
Weighted k-means / mini-batch k-means
Wrapper methods around supervised methods to incorporate unsupervised data
Wrapper methods + multiple views around supervised methods
SVD
Here's some of the other methods that we did not have the time and/or computational resources to explore, but we wanted to and are curious to see if other contestants gave them a shot:
Sparse autoencoders
Deep belief networks
Restricted boltzmann machines
Latent dirichlet allocation
Self-organizing maps
Graphical models