{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Baseline for Problem Set 2","metadata":{}},{"cell_type":"markdown","source":"The following notebook is a simple baseline (a lot of room for improvements) which can help guide you throughout the process of making a Convolutional Neural Network (CNN) for detecting features of a celebrity.\nWhile this notebook would attempt to explain CNNs in a easy-to-understand way, it is helpful if the reader has an understanding of how Neural Networks work. This video series by Stat Quest: https://bit.ly/3fXkbNA goes into great detail about Neural Networks.\n\nAdditionally, refer to the https://www.tisbmlthon.com/resources web-page for helpful starter resources. Finally, the documentation of each library used here (tensorflow.keras, pandas and to a lesser extent sklearn, matplotlib and seaborn) is also helpful for knowing the specifics of how a function work, but this notebook shall attempt to explain unfamiliar functions.\n\nContent: Overall, there are 4,600,000 images of celebrities (approximately 2,300,000 for training and 2,300,000 for testing) with the target attribute labelled: age\n\nData Files:\n\ncelebrities_crop: All 4,600,000 images of celebrities, the images are cropped such that only the face is in the frame\n\ncelebrities_train.csv: Attribute all features for our face images. Age being one of the features.","metadata":{}},{"cell_type":"code","source":"# Core libraries. Tensorflow for making Neural Networks\n\nimport numpy as np\nimport pandas as pd\nimport tensorflow as tf\nfrom tensorflow.keras import Model\nfrom tensorflow.keras.layers import Dense, Flatten, Input, Dropout\nfrom tensorflow.keras.applications import VGG16\nfrom tensorflow.keras.preprocessing.image import ImageDataGenerator\nfrom tensorflow.keras.callbacks import *\n\nfrom sklearn.model_selection import train_test_split\n\n# Data Visualization\nimport matplotlib.pyplot as plt\nimport seaborn as sns","metadata":{"execution":{"iopub.status.busy":"2022-04-10T10:21:46.418993Z","iopub.execute_input":"2022-04-10T10:21:46.419584Z","iopub.status.idle":"2022-04-10T10:21:46.425363Z","shell.execute_reply.started":"2022-04-10T10:21:46.419518Z","shell.execute_reply":"2022-04-10T10:21:46.424604Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Define the folder where all the images are there\nfolder = '../input/age-prediction/celebrities_crop_train/celebrities_crop_train'\n\n# Reading the train data\ntrain = pd.read_csv('../input/age-prediction/celebrities_final_train.csv')","metadata":{"execution":{"iopub.status.busy":"2022-04-10T10:23:57.749431Z","iopub.execute_input":"2022-04-10T10:23:57.749732Z","iopub.status.idle":"2022-04-10T10:23:58.300263Z","shell.execute_reply.started":"2022-04-10T10:23:57.7497Z","shell.execute_reply":"2022-04-10T10:23:58.2996Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Data Exploration","metadata":{}},{"cell_type":"markdown","source":"First let's understand the format of the training data, and how the variables are distributed.\n\nThe \"id\" column should give the name of the image file\n\nThe age of the celebrity isn't given explicitly, but that can be calculated from subtracting the year the photo was taken from the year of birth.","metadata":{}},{"cell_type":"code","source":"train","metadata":{"execution":{"iopub.status.busy":"2022-04-10T10:23:33.209125Z","iopub.execute_input":"2022-04-10T10:23:33.209391Z","iopub.status.idle":"2022-04-10T10:23:33.223406Z","shell.execute_reply.started":"2022-04-10T10:23:33.209363Z","shell.execute_reply":"2022-04-10T10:23:33.222585Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We can now visualise the distribution of the ages of the celebrities.","metadata":{}},{"cell_type":"code","source":"plt.hist(np.array(train['age']))","metadata":{"execution":{"iopub.status.busy":"2022-04-10T10:23:42.946491Z","iopub.execute_input":"2022-04-10T10:23:42.946791Z","iopub.status.idle":"2022-04-10T10:23:43.161319Z","shell.execute_reply.started":"2022-04-10T10:23:42.94676Z","shell.execute_reply":"2022-04-10T10:23:43.160382Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We can see that there are many more instances for ages between 30-50, but this is alright because that is the case for pictures of most celebrities nowadays.","metadata":{}},{"cell_type":"markdown","source":"Now, let us take a look at one of the images.","metadata":{}},{"cell_type":"code","source":"img = plt.imread(folder+\"/\"+train['id'][1])\nprint(\"Image dimensions\")\nprint(img.shape)\nplt.imshow(img)","metadata":{"execution":{"iopub.status.busy":"2022-04-10T10:24:18.356679Z","iopub.execute_input":"2022-04-10T10:24:18.356929Z","iopub.status.idle":"2022-04-10T10:24:18.383513Z","shell.execute_reply.started":"2022-04-10T10:24:18.356902Z","shell.execute_reply":"2022-04-10T10:24:18.382552Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"So each image is 50 x 50 pixels. The 3 in the last column is because it is an RGB image, so has 3 color channels. A black and white one would have only 1 channel.\n\nNow let's move on to modelling.","metadata":{}},{"cell_type":"markdown","source":"# Modelling","metadata":{}},{"cell_type":"markdown","source":"Since this is a medium size image with a lot of detail and is colored, we have to resort to transfer learning. Training a custom model would likely require days of training and much more data than provided.\n\nLet's first split the data into train and validation. Since we have massive amounts of data, we can leave only about 10% for validation, and use the rest to our benefit for training:","metadata":{}},{"cell_type":"code","source":"train_df, val_df = train_test_split(train, test_size=0.1)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Generators","metadata":{}},{"cell_type":"markdown","source":"Generators essentially help in reading in data and processing it appropriately. Since our images are in files, it would be more convenient to use a generator. Moreover, this builds a good pipeline as we can easily add Data Augmentation later on. Let's create the train and validation images","metadata":{}},{"cell_type":"code","source":"datagen = ImageDataGenerator(rescale=1./255.)\n\ncolumns = ['age']\n\ntrain_generator =       datagen.flow_from_dataframe(train_df, \n                                                    folder, \n                                                    x_col = 'id', \n                                                    y_col = columns, \n                                                    class_mode='raw',\n                                                    target_size=(50, 50), \n                                                    batch_size = 10, \n                                                    color_mode = 'rgb', \n                                                    shuffle = True)\n\nval_generator =   datagen.flow_from_dataframe(val_df, \n                                                    folder, \n                                                    x_col = 'id', \n                                                    y_col = columns, \n                                                    target_size =(50, 50), \n                                                    class_mode = 'raw', \n                                                    batch_size = 10, \n                                                    color_mode = 'rgb', \n                                                    shuffle = True)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Defining the model","metadata":{}},{"cell_type":"markdown","source":"We finally build the CNN in this step. For this particular model, we are basing our final model on VGG16. It is a powerful neural network which is easy to implement. This is also known as transfer learning as we are re-using a different model developed for a task as a base model. VGG16 has been trained on millions of images, and has become very good at understanding objects, in terms of the objects in the scene.\n\nWe shall add our own new layers at the end, as we want to channel this \"understanding\" of an image to predict variables specific to our problem.","metadata":{}},{"cell_type":"code","source":"def create_model():\n    # Input dimensions of our image (checked above)\n    x_input = Input(shape=(150, 150, 3))\n    \n    # VGG16 network\n    vgg = VGG16(weights='imagenet', include_top=False, input_shape=(150, 150, 3))\n    \n    x = vgg(x_input)\n    \n    # The output of a VGG network is also 3 dimensional. We need to convert it to 1 dimension and then to 2 outputs\n    \n    x = Flatten()(x) #Converting to one dimension by concatenating everything\n    \n    x = Dense(512, activation='relu')(x)\n    x = Dropout(0.2)(x) # Dropout randomly drops neurons in the network to prevent overfitting. This is kind of an arbitrary value\n    x = Dense(128, activation='relu')(x)\n    x = Dense(2, activation='sigmoid')(x)\n    \n    # Define our full model will inputs and outputs\n    model = Model(inputs=x_input, outputs = x)\n    \n    # We will be using Adam, it is the most common optimizer. This also has hyperparameters, but we are ignoring it for now.\n    # Binary Cross Entropy (log loss) is the metric we are optimizing. For our understanding, we are tracking accuracy and AUC (ignore this if you don't know it). Logloss is still the priority\n    model.compile(optimizer = \"Adam\", loss='binary_crossentropy', metrics=['accuracy', \"AUC\"])\n    \n    return model","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = create_model()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now let's train the model. Training the model is extremely slow on CPU, so you should try GPUs for this part. You can look at Colab or Kaggle Notebooks (30 hrs a week) for free GPUs.","metadata":{}},{"cell_type":"code","source":"# Fitting the model on the data.\n# Epochs is the number of times the model goes through the entire dataset. \n# You should choose a higher value to give it more time to learn. But too high a value can lead to memorization rather than learning.\n\nhistory = model.fit(train_generator, epochs=5, validation_data = val_generator)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['loss'])\nplt.plot(history.history['val_loss'])\nplt.title('Model Loss')\nplt.ylabel('Loss')\nplt.xlabel('Epoch')\nplt.legend(['train', 'validation'], loc='upper left')\nplt.show()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['accuracy'])\nplt.plot(history.history['val_accuracy'])\nplt.title('Model Accuracy')\nplt.ylabel('Accurcy')\nplt.xlabel('Epoch')\nplt.legend(['train', 'validation'], loc='upper left')\nplt.show()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Predictions","metadata":{}},{"cell_type":"markdown","source":"For making predictions, let's read in the sample Submission file to get the image Ids.","metadata":{}},{"cell_type":"code","source":"sub = pd.read_csv(\".../sample_submission.csv\")","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now let's create the test generator.\n\nNote that here:\n\nClass mode is None (we are predicting, not evaluating)\ny_col is None (we don't have true labels)\nEverything else should be the same.","metadata":{}},{"cell_type":"code","source":"test_generator =   datagen.flow_from_dataframe(sub, folder, x_col = 'full_path', y_col=None,\n    batch_size=,\n    class_mode=None,\n    target_size =(50, 50),  \n    shuffle=False,\n    color_mode = 'rgb')","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Making predictions and storing it in a variable.\npreds = model.predict(test_generator)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub['age'] = preds","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now let's convert the dataframe to a csv file, which you can download and submit on kaggle to get a score!","metadata":{}},{"cell_type":"code","source":"sub.to_csv(\"submission.csv\", index=False)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Going Forward","metadata":{}},{"cell_type":"markdown","source":"Tuning Hyperparameters (the hyperparameters we used were default or arbitrary). You should experiment with different values and use the validation data and Kaggle Leaderboard to compare different values.\nTry different model architecture. \n\nWe used VGG 16, but there is a long list of architectures you can use. Look here for a small list of models: https://www.tensorflow.org/api_docs/python/tf/keras/applications/\n\nData Augmentation - Here, you essentially morph the image a bit to create a new image (say changing zoom, etc). It is sometimes almost as good as if you had gotten extra data of new images. It helps the model become more robust to different face angles, etc.","metadata":{}}]}