# second place solution

First of all thanks Avito and Kaggle for the very interesting and challenging (especially for our hardware) competition. It was very pleasurable to compete here.
Features
Image Features: we extracted vectors (like in public kernels) from the pretrained VGG16, ImageNet, ResNet50 and MobileNet models.
Text Features: we trained Fasttext model on the full dataset and used it to generate vectors for title, description, title-city interaction, title-category interaction, stemmed title, stemmed descriptions. We used the same models for embeddings in our NN model.
Statistical Features: it was the most important type of numerical features that we had. We calculated average prices for each categorical feature and for second and third order of interactions. We calculated the average number of days the each advertisement had been active. Additionally, we calculated the same set of statistical features for each day during train and test periods.
Unsupervised Learning: we extracted vectors from autoencoder on categorical features. We trained a user2vec model to represent a user_id as a composition of other features.
Models
Neural Networks: Our best single model (0.2163 on public) is a neural network with different branches: FM like style for categorical features with embeddings, numerical features, concatenated fasttext vectors, concatenated image vectors, BiLSTM for words and BiLSTM for characters with concatenated max,avg poolings with attention (like we had in toxic competition), target encoded features for categorical features and their second and third order interactions, users 2 vectors features. Some details: cyclic LR, Nadam optimizer, plenty of BNs, big dropouts. Almost each of the branches have a dense layer before concatenating them.
LightGBM: Surprisingly, fasttext vectors were helpful for lightgbm single model as well (0.2188 on public). SVD over TFIDF transformation was a nice feature for our models. Numerical features and target encoded categories made the rest of the work.
Bunch of weak models: FM_FTRL, Ridge, CatBoost.
Stack: we developed six layers (OMG) stack. 1,2,3,4 and 6 layer have been made with lightgbm and all previous metafeatures. We used unique meta information for each layer, based on a weaknesses of the previous layer.
Validation
10 folds CV was the best choice for us. It was accurate and fast enough.