{"cells":[{"metadata":{},"cell_type":"markdown","source":"## Shap Deep Explainer\n\n![MNIST](https://storage.googleapis.com/kaggle-media/competitions/Kannada-MNIST/kannada.png)\n\nCNN Code from [[Pytorch] 1. ANN & Simple CNN](https://www.kaggle.com/subinium/pytorch-1-ann-simple-cnn) kernel."},{"metadata":{},"cell_type":"markdown","source":"## Simple CNN with Pytorch"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import torch, torchvision\nfrom torchvision import datasets, transforms\nfrom torch import nn, optim\nfrom torch.nn import functional as F\nimport pandas as pd\nfrom sklearn.model_selection import train_test_split\nfrom torch.autograd import Variable\nimport numpy as np\nimport shap","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train = pd.read_csv(\"../input/Kannada-MNIST/train.csv\",dtype = np.float32)\n\n\ntarget = train.label.values\ntrain = train.loc[:,train.columns != \"label\"].values/255 \n\nX_train, X_test, y_train, y_test = train_test_split(train, target, test_size = 0.2, random_state = 42) \n\nX_train = torch.from_numpy(X_train)\ny_train = torch.from_numpy(y_train).type(torch.LongTensor) # data type is long\n\nX_test = torch.from_numpy(X_test)\ny_test = torch.from_numpy(y_test).type(torch.LongTensor) # data type is long\n\nbatch_size = 128\nnum_epochs = 100\n\ntrain = torch.utils.data.TensorDataset(X_train, y_train)\ntest = torch.utils.data.TensorDataset(X_test, y_test)\n\ntrain_loader = torch.utils.data.DataLoader(train, batch_size = batch_size, shuffle = False)\ntest_loader = torch.utils.data.DataLoader(test, batch_size = batch_size, shuffle = False)\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### CNN Model Class & Train"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"class CNNModel(nn.Module):\n    def __init__(self):\n        super(CNNModel, self).__init__()\n        self.cnn1 = nn.Conv2d(in_channels=1, out_channels=16, kernel_size=5, stride=1, padding=0)\n        self.relu1 = nn.ReLU()\n        self.maxpool1 = nn.MaxPool2d(kernel_size=2)\n        self.cnn2 = nn.Conv2d(in_channels=16, out_channels=32, kernel_size=5, stride=1, padding=0)\n        self.relu2 = nn.ReLU()\n        self.maxpool2 = nn.MaxPool2d(kernel_size=2)\n        self.fc1 = nn.Linear(32 * 4 * 4, 10) \n    \n    def forward(self, x):\n        out = self.cnn1(x)\n        out = self.relu1(out)\n        out = self.maxpool1(out)\n        out = self.cnn2(out)\n        out = self.relu2(out)\n        out = self.maxpool2(out)\n        out = out.view(out.size(0), -1)\n        out = self.fc1(out)\n        return out\n    \n# Create CNN\nmodel = CNNModel()\n\n# Cross Entropy Loss \nerror = nn.CrossEntropyLoss()\n\n# SGD Optimizer\nlearning_rate = 0.1\noptimizer = torch.optim.SGD(model.parameters(), lr=learning_rate)\n\n# CNN model training\ncount = 0\nloss_list = []\niteration_list = []\naccuracy_list = []\n\nfor epoch in range(num_epochs):\n    for i, (images, labels) in enumerate(train_loader):\n\n        train = Variable(images.view(-1, 1, 28, 28))\n        labels = Variable(labels)\n        \n        \n        optimizer.zero_grad() # Clear gradients\n        outputs = model(train) # Forward propagation\n        loss = error(outputs, labels) # Calculate softmax and cross entropy loss\n        loss.backward() # Calculating gradients\n        optimizer.step() # Update parameters\n        \n        count += 1\n        \n        if count % 50 == 0:\n            # Calculate Accuracy         \n            correct = 0\n            total = 0\n            \n            # Predict test dataset\n            for images, labels in test_loader:\n                test = Variable(images.view(-1, 1, 28, 28))\n                outputs = model(test) # Forward propagation\n                predicted = torch.max(outputs.data, 1)[1] # Get predictions from the maximum value\n                total += len(labels) # Total number of labels\n                correct += (predicted == labels).sum() # Total correct predictions\n            \n            accuracy = 100.0 * correct.item() / total\n            \n            # store loss and iteration\n            loss_list.append(loss.data.item())\n            iteration_list.append(count)\n            accuracy_list.append(accuracy)\n            if count % 500 == 0:\n                print('Iteration: {}  Loss: {}  Accuracy: {} %'.format(count, loss.data.item(), accuracy))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Shap Value : Deep Explainer\n\n[official documentation](https://shap.readthedocs.io/en/latest/#shap.DeepExplainer)\n\nMeant to approximate SHAP values for deep learning models.\n\nThis is an enhanced version of the DeepLIFT algorithm (Deep SHAP) where, similar to Kernel SHAP, we approximate the conditional expectations of SHAP values using a selection of background samples. Lundberg and Lee, NIPS 2017 showed that the per node attribution rules in DeepLIFT (Shrikumar, Greenside, and Kundaje, arXiv 2017) can be chosen to approximate Shapley values. By integrating over many backgound samples DeepExplainer estimates approximate SHAP values such that they sum up to the difference between the expected model output on the passed background samples and the current model output (f(x) - E[f(x)]).\n\n**`shap_values(X, ranked_outputs=None, output_rank_order='max')`**\n\nReturn approximate SHAP values for the model applied to the data given by X.\n\n**`X :list,`**\n\nif framework == ‘tensorflow’: numpy.array, or pandas.DataFrame if framework == ‘pytorch’: torch.tensor A tensor (or list of tensors) of samples (where X.shape[0] == # samples) on which to explain the model’s output.\n\n\n**`ranked_outputs :None or int`**\n\nIf ranked_outputs is None then we explain all the outputs in a multi-output model. If ranked_outputs is a positive integer then we only explain that many of the top model outputs (where “top” is determined by output_rank_order). Note that this causes a pair of values to be returned (shap_values, indexes), where shap_values is a list of numpy arrays for each of the output ranks, and indexes is a matrix that indicates for each sample which output indexes were choses as “top”.\n\n**`output_rank_order :“max”, “min”, or “max_abs”`**\n\nHow to order the model outputs when using ranked_outputs, either by maximum, minimum, or maximum absolute value.\nFor a models with a single output this returns a tensor of SHAP values with the same shape as X. For a model with multiple outputs this returns a list of SHAP value tensors, each of which are the same shape as X. If ranked_outputs is None then this list of tensors matches the number of model outputs. If ranked_outputs is a positive integer a pair is returned (shap_values, indexes), where shap_values is a list of tensors with a length of ranked_outputs, and indexes is a matrix that indicates for each sample which output indexes were chosen as “top”."},{"metadata":{"trusted":true},"cell_type":"code","source":"batch = next(iter(test_loader))\nimages, _ = batch\nimages = images.view(-1, 1, 28, 28)\n\nbackground = images[:100]\ntest_images= images[100:110]\n\ne = shap.DeepExplainer(model, images)\nshap_values = e.shap_values(test_images)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"shap_numpy = [np.swapaxes(np.swapaxes(s, 1, -1), 1, 2) for s in shap_values]\ntest_numpy = np.swapaxes(np.swapaxes(test_images.numpy(), 1, -1), 1, 2)\n\nshap.image_plot(shap_numpy, -test_numpy)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(_[100:110])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The plot above shows the explanations for each class on four predictions. Note that the explanations are ordered for the classes 0-9 going left to right along the rows."},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}