Credits to Jim Zhang for helping with these tips

## Anaconda
If using Python, pandas is recommended for data manipulation. For prediction, one can start with some typical models in scikit-learn. pandas, numpy, sklearn are all included in Anaconda.

## Useful pandas functions
```
pd.read_csv('filename.csv', index_col='id')
pd.DataFrame.to_csv('submission.csv', index_label='id')
df['col_name'].replace(['option_a', 'option_b'], [0, 1], inplace=True)
pd.to_numeric(df)
df.index.names = ['id']
```

## Useful sklearn functions
* [SimpleImputer](https://scikit-learn.org/stable/modules/generated/sklearn.impute.SimpleImputer.html)
* [AUC Score](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_auc_score.html) 

## Start simple
Start with a minimal viable preprocessing+classification [pipeline](https://scikit-learn.org/stable/modules/generated/sklearn.pipeline.Pipeline.html#sklearn.pipeline.Pipeline). This will give you a baseline to compare against as you try different approaches.

## But don't be afraid to try things you haven't heard of before
After some basic models, one can progress to some more advanced models in other libraries. Maybe search around a bit for cool or successful models that people have used for classification.

## IDE/Shortcuts
Learn all the shortcuts to your development environment. Spyder and Jupyter are both included in Anaconda, and there are plenty of shortcuts to run specific code sections swiftly.

## Missing data
Think of how to fill in the missing/nan data. A starting point is `SimpleImputer`.

## Feature engineering
See if you can create new features out of the existing columns, that would make the model perform better / have an easier time fitting the data

## Model blending
Why use one model when you can take the average of multiple model outputs, or even use another model on top of the model outputs?

## Probability calibration
For probabilistic models, use the `.predict_proba(X)[:,1]` method and column-indexing to output the probability of class 1.

Some models don't have accurate classification probabilities, but one can calibrate them using various techniques. This doesn't help much for individual models due to the AUC metric, but allows for "fairer" model blending if you plan to take a simple average.

## Hyperparameter tuning
Gain some intuition with validation curves and then try some automatic tuners! Be aware, poor model types usually will not improve much with tuning.