{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"Forked from https://www.kaggle.com/alpaca0984/dog-vs-cat-with-pytorch","metadata":{}},{"cell_type":"markdown","source":"# References\n\n- The competition is https://www.kaggle.com/c/dogs-vs-cats-redux-kernels-edition (it ended long time ago)\n- Pytorch official docs:\n    - [Finetuning Torchvision Models](https://pytorch.org/tutorials/beginner/finetuning_torchvision_models_tutorial.html)\n    - [Writing Custom Datasets, DataLoaders and Transforms](https://pytorch.org/tutorials/beginner/data_loading_tutorial.html)\n    - Any other docs refered to train model or whatever, left comments on each section.\n- Refered kernels:\n    - [Dog_vs_Cat Transfer Learning - VGG16 by Pytorch](https://www.kaggle.com/bootiu/dog-vs-cat-transfer-learning-vgg16-by-pytorch)","metadata":{"id":"b7OkxS36u3_C"}},{"cell_type":"markdown","source":"# Import libraries","metadata":{"id":"BtnC7L1DjJJe"}},{"cell_type":"code","source":"# Standard library\nimport copy\nimport glob\nimport multiprocessing\nimport os\nimport time\nimport zipfile\n\n# Pytorch\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torch.optim as optim\nfrom torch.utils.data import Dataset, DataLoader\nfrom torchvision import models, transforms\n\n# Related third party\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport pandas as pd\nfrom PIL import Image\nfrom skimage import io, transform\nfrom sklearn.model_selection import train_test_split\nfrom tqdm.notebook import tqdm","metadata":{"id":"EIYPRiPDTI71","execution":{"iopub.status.busy":"2021-07-02T20:52:46.452341Z","iopub.execute_input":"2021-07-02T20:52:46.452694Z","iopub.status.idle":"2021-07-02T20:52:48.911469Z","shell.execute_reply.started":"2021-07-02T20:52:46.452657Z","shell.execute_reply":"2021-07-02T20:52:48.910695Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Pre process for each environment","metadata":{"id":"jYmov1HJIDCJ"}},{"cell_type":"markdown","source":"### For Kaggle kernel","metadata":{"id":"ZI4kPwGkH8ID"}},{"cell_type":"code","source":"base_dir = '../input/dogs-vs-cats-redux-kernels-edition'\nwith zipfile.ZipFile(os.path.join(base_dir, 'train.zip')) as train_zip:\n    train_zip.extractall('../data')\nwith zipfile.ZipFile(os.path.join(base_dir, 'test.zip')) as test_zip:\n    test_zip.extractall('../data')\n\ntrain_dir = '../data/train'\ntest_dir = '../data/test'","metadata":{"id":"r1Gm9UbE2HVS","execution":{"iopub.status.busy":"2021-07-02T20:52:48.914304Z","iopub.execute_input":"2021-07-02T20:52:48.914561Z","iopub.status.idle":"2021-07-02T20:53:07.596802Z","shell.execute_reply.started":"2021-07-02T20:52:48.914536Z","shell.execute_reply":"2021-07-02T20:53:07.595999Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### For Google Colab\n\nIt assumes that downloaded data will be located under `'My Drive/Data Set/'`","metadata":{"id":"EEdVm4F1H-tC"}},{"cell_type":"code","source":"# with zipfile.ZipFile('./drive/My Drive/Data Set/dogs-vs-cats-redux-kernels-edition.zip') as entire_zip:\n#     entire_zip.extractall('.')\n# with zipfile.ZipFile('./train.zip') as train_zip:\n#     train_zip.extractall('.')\n# with zipfile.ZipFile('./test.zip') as test_zip:\n#     test_zip.extractall('.')\n\n# train_dir = './train'\n# test_dir = './test'","metadata":{"id":"axQSzMXeH-Wz","execution":{"iopub.status.busy":"2021-07-02T20:53:07.598077Z","iopub.execute_input":"2021-07-02T20:53:07.598409Z","iopub.status.idle":"2021-07-02T20:53:07.605942Z","shell.execute_reply.started":"2021-07-02T20:53:07.598373Z","shell.execute_reply":"2021-07-02T20:53:07.604998Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Global Declarations","metadata":{"id":"W91auWwe1M6R"}},{"cell_type":"markdown","source":"## Constants\n\nRegarding `input_size`, `mean` and `std`, all pre-trained models expect input images normalized in the same way, i.e. mini-batches of 3-channel RGB images of shape (3 x H x W), where H and W are expected to be at least 224. The images have to be loaded in to a range of [0, 1] and then normalized using `mean = [0.485, 0.456, 0.406]` and `std = [0.229, 0.224, 0.225]`. See [torchvision.models](https://pytorch.org/docs/stable/torchvision/models.html) for details. ","metadata":{"id":"AAit-F1ijnY_"}},{"cell_type":"code","source":"input_size = 224\nmean = [0.485, 0.456, 0.406]\nstd = [0.229, 0.224, 0.225]\n\n# Number of classes in the dataset\nnum_classes = 2 # dog, cat\n\n# Batch size for training (change depending on how much memory you have)\nbatch_size = 32\n\n# Number of epochs to train for\nnum_epochs = 2\n\n# Flag for feature extracting. When False, we finetune the whole model,\n#   when True we only update the reshaped layer params\nfeature_extract = True\n\n# Switch to perform multi-process data loading\nnum_workers = multiprocessing.cpu_count()","metadata":{"id":"giTRYBjlZQQM","execution":{"iopub.status.busy":"2021-07-02T20:53:07.607241Z","iopub.execute_input":"2021-07-02T20:53:07.607719Z","iopub.status.idle":"2021-07-02T20:53:07.627418Z","shell.execute_reply.started":"2021-07-02T20:53:07.607681Z","shell.execute_reply":"2021-07-02T20:53:07.626726Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Helper Functions","metadata":{"id":"SQkSsiGQjRUR"}},{"cell_type":"code","source":"# train data file looks './train/dog.10435.jpg'\n# test data file looks './test/10435.jpg'\ndef extract_class_from(path):\n    file = path.split('/')[-1]\n    return file.split('.')[0]","metadata":{"id":"J5MyzmeIfQU4","execution":{"iopub.status.busy":"2021-07-02T20:53:07.631227Z","iopub.execute_input":"2021-07-02T20:53:07.63163Z","iopub.status.idle":"2021-07-02T20:53:07.636606Z","shell.execute_reply.started":"2021-07-02T20:53:07.631593Z","shell.execute_reply":"2021-07-02T20:53:07.635797Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This `train_model` function comes from https://pytorch.org/tutorials/beginner/finetuning_torchvision_models_tutorial.html#model-training-and-validation-code.","metadata":{"id":"MYI0tW1Cx6Dg"}},{"cell_type":"code","source":"def train_model(model, dataloaders, criterion, optimizer, num_epochs=25):\n    since = time.time()\n\n    history = {'accuracy': [],\n               'val_accuracy': [],\n               'loss': [],\n               'val_loss': []}\n\n    best_model_wts = copy.deepcopy(model.state_dict())\n    best_acc = 0.0\n\n    for epoch in range(num_epochs):\n        print('Epoch {}/{}'.format(epoch, num_epochs - 1))\n        print('-' * 10)\n\n        # Each epoch has a training and validation phase\n        for phase in ['train', 'val']:\n            if phase == 'train':\n                model.train()  # Set model to training mode\n            else:\n                model.eval()   # Set model to evaluate mode\n\n            running_loss = 0.0\n            running_corrects = 0\n\n            # Iterate over data.\n            for inputs, labels in tqdm(dataloaders[phase]):\n                inputs = inputs.to(device)\n                labels = labels.to(device)\n\n                # zero the parameter gradients\n                optimizer.zero_grad()\n\n                # forward\n                # track history if only in train\n                with torch.set_grad_enabled(phase == 'train'):\n                    outputs = model(inputs)\n                    loss = criterion(outputs, labels)\n\n                    _, preds = torch.max(outputs, 1)\n\n                    # backward + optimize only if in training phase\n                    if phase == 'train':\n                        loss.backward()\n                        optimizer.step()\n\n                # statistics\n                running_loss += loss.item() * inputs.size(0)\n                running_corrects += torch.sum(preds == labels.data)\n\n            epoch_loss = running_loss / len(dataloaders[phase].dataset)\n            epoch_acc = running_corrects.double() / len(dataloaders[phase].dataset)\n\n            print('{} Loss: {:.4f} Acc: {:.4f}'.format(phase, epoch_loss, epoch_acc))\n\n            # deep copy the model\n            if phase == 'val' and epoch_acc > best_acc:\n                best_acc = epoch_acc\n                best_model_wts = copy.deepcopy(model.state_dict())\n\n            if phase == 'train':\n                history['accuracy'].append(epoch_acc.item())\n                history['loss'].append(epoch_loss)\n            else:\n                history['val_accuracy'].append(epoch_acc.item())\n                history['val_loss'].append(epoch_loss) \n\n        print()\n\n    time_elapsed = time.time() - since\n    print('Training complete in {:.0f}m {:.0f}s'.format(time_elapsed // 60, time_elapsed % 60))\n    print('Best val Acc: {:4f}'.format(best_acc))\n\n    # load best model weights\n    model.load_state_dict(best_model_wts)\n    return model, history","metadata":{"id":"lBTPEedns0ZV","execution":{"iopub.status.busy":"2021-07-02T20:53:07.638744Z","iopub.execute_input":"2021-07-02T20:53:07.639113Z","iopub.status.idle":"2021-07-02T20:53:07.658393Z","shell.execute_reply.started":"2021-07-02T20:53:07.639077Z","shell.execute_reply":"2021-07-02T20:53:07.657496Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Load Data","metadata":{"id":"bOUrAeG5X5pt"}},{"cell_type":"code","source":"all_train_files = glob.glob(os.path.join(train_dir, '*.jpg'))\ntrain_list, val_list = train_test_split(all_train_files, random_state=42)","metadata":{"id":"bLNGKWkeX7Xt","execution":{"iopub.status.busy":"2021-07-02T20:53:07.659808Z","iopub.execute_input":"2021-07-02T20:53:07.660311Z","iopub.status.idle":"2021-07-02T20:53:07.757978Z","shell.execute_reply.started":"2021-07-02T20:53:07.660273Z","shell.execute_reply":"2021-07-02T20:53:07.757336Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(len(train_list))\nprint(len(val_list))","metadata":{"id":"6gcueQhGfVpb","outputId":"318760a0-c18e-4314-bb74-9bc3fe879046","execution":{"iopub.status.busy":"2021-07-02T20:53:07.760315Z","iopub.execute_input":"2021-07-02T20:53:07.760788Z","iopub.status.idle":"2021-07-02T20:53:07.768049Z","shell.execute_reply.started":"2021-07-02T20:53:07.760747Z","shell.execute_reply":"2021-07-02T20:53:07.767061Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Check what train data looks like","metadata":{"id":"ql8hbhRVv9ot"}},{"cell_type":"code","source":"fig, axes = plt.subplots(nrows=2,\n                         ncols=3,\n                         figsize=(18, 12))\nfor img_path, ax in zip(train_list, axes.ravel()):\n    ax.set_title(img_path)\n    ax.imshow(Image.open(img_path))","metadata":{"id":"DUE_m0dMwBH2","outputId":"2fe6ef81-d235-4608-8c9d-080943522263","execution":{"iopub.status.busy":"2021-07-02T20:53:07.769415Z","iopub.execute_input":"2021-07-02T20:53:07.769687Z","iopub.status.idle":"2021-07-02T20:53:08.932473Z","shell.execute_reply.started":"2021-07-02T20:53:07.769651Z","shell.execute_reply":"2021-07-02T20:53:08.931434Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Dataset class\n\nSee https://pytorch.org/tutorials/beginner/data_loading_tutorial.html#dataset-class for details.","metadata":{"id":"RRE7mKb1b3SW"}},{"cell_type":"code","source":"class DogVsCatDataset(Dataset):\n  \n    def __init__(self, file_list, transform=None):\n        self.file_list = file_list\n        self.transform = transform\n    \n    def __len__(self):\n        return len(self.file_list)\n  \n    def __getitem__(self, idx):\n        if torch.is_tensor(idx):\n            idx = idx.tolist()\n       \n        img_name = self.file_list[idx]\n        image = Image.open(img_name)\n        if self.transform:\n            image = self.transform(image)\n    \n        label_category = extract_class_from(img_name)\n        label = 1 if label_category == 'dog' else 0\n    \n        return image, label","metadata":{"id":"2VFbJ_RahO0T","execution":{"iopub.status.busy":"2021-07-02T20:53:08.933719Z","iopub.execute_input":"2021-07-02T20:53:08.93404Z","iopub.status.idle":"2021-07-02T20:53:08.943934Z","shell.execute_reply.started":"2021-07-02T20:53:08.933998Z","shell.execute_reply":"2021-07-02T20:53:08.943063Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Create dataloaders\n\nSee https://pytorch.org/tutorials/beginner/finetuning_torchvision_models_tutorial.html#load-data for details.","metadata":{"id":"TaDWc8tobuZ_"}},{"cell_type":"code","source":"data_transforms = {\n    'train': transforms.Compose([\n        transforms.RandomResizedCrop(input_size, scale=(0.5, 1.0)),\n        transforms.RandomHorizontalFlip(),\n        transforms.ToTensor(),\n        transforms.Normalize(mean, std)\n    ]),\n    'val': transforms.Compose([\n        transforms.Resize(input_size),\n        transforms.CenterCrop(input_size),\n        transforms.ToTensor(),\n        transforms.Normalize(mean, std)\n    ])\n}","metadata":{"id":"lNaNM8X-Qgks","execution":{"iopub.status.busy":"2021-07-02T20:53:08.945416Z","iopub.execute_input":"2021-07-02T20:53:08.945921Z","iopub.status.idle":"2021-07-02T20:53:08.95388Z","shell.execute_reply.started":"2021-07-02T20:53:08.945883Z","shell.execute_reply":"2021-07-02T20:53:08.952965Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Create training and validation datasets\nimage_datasets = {\n    'train': DogVsCatDataset(train_list,\n                             transform=data_transforms['train']),\n    'val': DogVsCatDataset(val_list,\n                           transform=data_transforms['val'])\n}\n\n# Create training and validation dataloaders\ndataloaders_dict = {x: DataLoader(image_datasets[x],\n                                  batch_size=batch_size,\n                                  shuffle=True,\n                                  num_workers=num_workers) for x in ['train', 'val']}\n\n# Detect if we have a GPU available\ndevice = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")","metadata":{"id":"JI12IHfujr_Y","execution":{"iopub.status.busy":"2021-07-02T20:53:08.955167Z","iopub.execute_input":"2021-07-02T20:53:08.955705Z","iopub.status.idle":"2021-07-02T20:53:08.98769Z","shell.execute_reply.started":"2021-07-02T20:53:08.955668Z","shell.execute_reply":"2021-07-02T20:53:08.986837Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Initialize and Reshape the Networks\n\nRegarding tuning VGG, see https://pytorch.org/tutorials/beginner/finetuning_torchvision_models_tutorial.html#vgg.\n\n","metadata":{"id":"zdXNdTZwcDVQ"}},{"cell_type":"code","source":"model_ft = models.mobilenet_v2(pretrained=True)\nmodel_ft.classifier[1] = nn.Linear(1280, num_classes)","metadata":{"id":"jRY3V7P-0D_K","execution":{"iopub.status.busy":"2021-07-02T20:57:49.445699Z","iopub.execute_input":"2021-07-02T20:57:49.446072Z","iopub.status.idle":"2021-07-02T20:57:49.557609Z","shell.execute_reply.started":"2021-07-02T20:57:49.446037Z","shell.execute_reply":"2021-07-02T20:57:49.556666Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Create the Optimizer\n\nSee https://pytorch.org/tutorials/beginner/finetuning_torchvision_models_tutorial.html#create-the-optimizer for details.","metadata":{"id":"do0wSVBeLYr5"}},{"cell_type":"code","source":"# Send the model to GPU\nmodel_ft = model_ft.to(device)\n\n# Gather the parameters to be optimized/updated in this run. If we are\n#  finetuning we will be updating all parameters. However, if we are\n#  doing feature extract method, we will only update the parameters\n#  that we have just initialized, i.e. the parameters with requires_grad\n#  is True.\nparams_to_update = model_ft.parameters()\nprint(\"Params to learn:\")\nif feature_extract:\n    params_to_update = []\n    for name,param in model_ft.named_parameters():\n        if param.requires_grad == True:\n            params_to_update.append(param)\n            print(\"\\t\",name)\nelse:\n    for name,param in model_ft.named_parameters():\n        if param.requires_grad == True:\n            print(\"\\t\",name)\n\n# Observe that all parameters are being optimized\noptimizer_ft = optim.SGD(params_to_update, lr=0.001, momentum=0.9)","metadata":{"id":"ko6DPWZ8xhKZ","outputId":"e77c16f6-e6fa-473f-eafa-9c2d484da1a3","execution":{"iopub.status.busy":"2021-07-02T20:57:52.268231Z","iopub.execute_input":"2021-07-02T20:57:52.268551Z","iopub.status.idle":"2021-07-02T20:57:52.306604Z","shell.execute_reply.started":"2021-07-02T20:57:52.268521Z","shell.execute_reply":"2021-07-02T20:57:52.305908Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Run Training and Validation Step","metadata":{"id":"MNuqAgyBdeIs"}},{"cell_type":"code","source":"# Setup the loss fxn\ncriterion = nn.CrossEntropyLoss()\n\n# Train and evaluate\nmodel_ft, hist = train_model(model_ft, dataloaders_dict, criterion, optimizer_ft, num_epochs=num_epochs)","metadata":{"id":"QFceXYTey2FI","outputId":"a08c06dd-62ae-4b01-b29d-50535d8f2288","execution":{"iopub.status.busy":"2021-07-02T20:57:58.234989Z","iopub.execute_input":"2021-07-02T20:57:58.235312Z","iopub.status.idle":"2021-07-02T21:03:25.648157Z","shell.execute_reply.started":"2021-07-02T20:57:58.235281Z","shell.execute_reply":"2021-07-02T21:03:25.647202Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Visualize training results\n\nThis procedure comes from https://www.tensorflow.org/tutorials/images/classification#visualize_training_results.","metadata":{"id":"SuaL38c3E1Wr"}},{"cell_type":"code","source":"acc = hist['accuracy']\nval_acc = hist['val_accuracy']\nloss = hist['loss']\nval_loss = hist['val_loss']\nepochs_range = range(num_epochs)\n\nplt.figure(figsize=(24, 8))\nplt.subplot(1, 2, 1)\nplt.plot(epochs_range, acc, label='Training Accuracy')\nplt.plot(epochs_range, val_acc, label='Validation Accuracy')\nplt.legend(loc='lower right')\nplt.title('Training and Validation Accuracy')\n\nplt.subplot(1, 2, 2)\nplt.plot(epochs_range, loss, label='Training Loss')\nplt.plot(epochs_range, val_loss, label='Validation Loss')\nplt.legend(loc='upper right')\nplt.title('Training and Validation Loss')\nplt.show()","metadata":{"id":"lCzmHtEfEwfc","outputId":"e9138e5f-748f-491d-90f2-54b0fe209f90","execution":{"iopub.status.busy":"2021-07-02T21:03:32.711762Z","iopub.execute_input":"2021-07-02T21:03:32.712106Z","iopub.status.idle":"2021-07-02T21:03:33.015357Z","shell.execute_reply.started":"2021-07-02T21:03:32.712068Z","shell.execute_reply":"2021-07-02T21:03:33.014285Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Predict","metadata":{"id":"toHjy4Iajf3V"}},{"cell_type":"code","source":"test_list = glob.glob(os.path.join(test_dir, '*.jpg'))\ntest_data_transform = data_transforms['val']\n\nids = []\nlabels = []\n\nwith torch.no_grad():\n    for test_path in tqdm(test_list):\n        img = Image.open(test_path)\n        img = test_data_transform(img)\n        img = img.unsqueeze(0)\n        img = img.to(device)\n\n        model_ft.eval()\n        outputs = model_ft(img)\n        preds = F.softmax(outputs, dim=1)[:, 1].tolist()\n\n        test_id = extract_class_from(test_path)\n        ids.append(int(test_id))\n        labels.append(preds[0])","metadata":{"id":"Sea_fIAsTyK-","outputId":"0c6458e3-db80-438f-a073-c0eb4aab077f","execution":{"iopub.status.busy":"2021-07-02T21:04:08.403027Z","iopub.execute_input":"2021-07-02T21:04:08.403361Z","iopub.status.idle":"2021-07-02T21:06:50.934465Z","shell.execute_reply.started":"2021-07-02T21:04:08.403332Z","shell.execute_reply":"2021-07-02T21:06:50.933593Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Check how well the prediction went","metadata":{"id":"NkKIJbYo5rKy"}},{"cell_type":"code","source":"template = '\"{}\" with {:.2%} confidence'\ndef pred_result_message(pred):\n    if pred > 0.5:\n        return template.format('dog', pred)\n    else:\n        return template.format('cat', 1 - pred)\n\nfig, axes = plt.subplots(nrows=2,\n                         ncols=3,\n                         figsize=(18, 12))\nfor img_path, label, ax in zip(test_list, labels, axes.ravel()):\n    ax.set_title(pred_result_message(label))\n    ax.imshow(Image.open(img_path))","metadata":{"id":"-WhwOwuN6X-_","outputId":"6dae7037-8584-4ee9-aa8d-cfdf3af61101","execution":{"iopub.status.busy":"2021-07-02T21:08:35.163842Z","iopub.execute_input":"2021-07-02T21:08:35.164175Z","iopub.status.idle":"2021-07-02T21:08:36.172001Z","shell.execute_reply.started":"2021-07-02T21:08:35.164146Z","shell.execute_reply":"2021-07-02T21:08:36.171206Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Generate submittion.csv","metadata":{"id":"EEBvETUujpZb"}},{"cell_type":"code","source":"output = pd.DataFrame({'id': ids,\n                       'label': np.round(labels)})\n\noutput.sort_values(by='id', inplace=True)\noutput.reset_index(drop=True, inplace=True)\n\noutput.to_csv('submission.csv', index=False)","metadata":{"id":"S0hI7a9GgLM_","execution":{"iopub.status.busy":"2021-07-02T21:08:44.891338Z","iopub.execute_input":"2021-07-02T21:08:44.891702Z","iopub.status.idle":"2021-07-02T21:08:45.152443Z","shell.execute_reply.started":"2021-07-02T21:08:44.891668Z","shell.execute_reply":"2021-07-02T21:08:45.151624Z"},"trusted":true},"execution_count":null,"outputs":[]}]}