{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# HotterDog, Theft and Salt","metadata":{}},{"cell_type":"markdown","source":"These challenges goal are all the same tricking a neural network to misclassify an image with as little perturbation as possible.\n\nThese challenges are also tightly bound together. The underlying model used for all of them is a tf.keras.applications.MobileNet with weights pretrained on imagenet dataset. Actually I never though about it at the beginning, but seeing the predicted classes trying several approaches I noticed a lot of overlap, so I tried out the network available in the Salt challenge on hotterdog and once I saw that the classes were overlapping I understood what was needed to do for all three these challenges ( even though I had already black box attacked successfully both theft and salt).\n\nIn this notebook I show the use of the [adversarial robustness toolbox](https://github.com/Trusted-AI/adversarial-robustness-toolbox)(art) library to perform a white box attack on the models.\n\nSo we can say that this notebook shows three cases of [Art Attack](https://www.youtube.com/watch?v=cFqCl_GvHe0).","metadata":{}},{"cell_type":"markdown","source":"## Provided inports","metadata":{}},{"cell_type":"code","source":"\nimport pandas as pd\nimport numpy as np\nimport os\nimport requests\nfrom IPython.display import Image as Ima\nfrom IPython.display import clear_output, display\nfrom PIL import Image as Im\nfrom torch.utils.data import Dataset, DataLoader\nimport torch\nimport joblib\nimport matplotlib.pyplot as plt\nimport json\nimport base64\nimport shutil\nimport warnings\nwarnings.filterwarnings(\"ignore\")\n\n\ndir_name = \"/kaggle/input/ai-village-ctf\"\n\n","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:07:44.476648Z","iopub.execute_input":"2022-09-11T21:07:44.477154Z","iopub.status.idle":"2022-09-11T21:07:46.497554Z","shell.execute_reply.started":"2022-09-11T21:07:44.477047Z","shell.execute_reply":"2022-09-11T21:07:46.496218Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def add_flag(challenge_id: str, flag: str) -> str:\n    try:\n        submission_path = \"/kaggle/working/submission.csv\"\n        template_path = f\"{dir_name}/submission.csv\"\n        if not os.path.isfile(submission_path) :\n            shutil.copyfile(template_path, submission_path)\n        flags = pd.read_csv(submission_path, index_col=\"challenge_id\")\n        valid = flags.index.to_list()\n        if challenge_id in valid:\n            flags.at[challenge_id] = flag\n            flags.to_csv(submission_path)\n            print(\"Flag saved. If you want to update the scoreboard, click 'submit' on the right. Remember that you can only update your score on the scoreboard 50 times per day\")\n        else:\n            print(\"Invalid challenge_id\")\n    except:\n        print(\"We had trouble saving that flag. Keep a copy of it and contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:07:46.499759Z","iopub.execute_input":"2022-09-11T21:07:46.500696Z","iopub.status.idle":"2022-09-11T21:07:46.514188Z","shell.execute_reply.started":"2022-09-11T21:07:46.500645Z","shell.execute_reply":"2022-09-11T21:07:46.512665Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# HOTTERDOG (id: hotterdog) -- 400 Points\n\n### Scenario\n\nChester submitted himself in the \"most creative hotdog\" contest. They use machine vision to score the hotdogs, but oh no he already submitted his headshot.\n\n### Objective\n\nHelp Chester classify as a hotdog, but it has to look like Chester to a human. Modify his picture to fool a classifier.","metadata":{}},{"cell_type":"code","source":"%%capture\n!pip install adversarial-robustness-toolbox\n!wget https://www.collinsdictionary.com/images/full/hotdog_156787409_1000.jpg -O hotdog_img.jpg","metadata":{"id":"zLTN_KEghcYG","execution":{"iopub.status.busy":"2022-09-11T21:07:46.515936Z","iopub.execute_input":"2022-09-11T21:07:46.516549Z","iopub.status.idle":"2022-09-11T21:08:02.175569Z","shell.execute_reply.started":"2022-09-11T21:07:46.516500Z","shell.execute_reply":"2022-09-11T21:08:02.174085Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#adversarial-robustness-toolbox used required disabling eager execution \nimport tensorflow as tf\nif tf.executing_eagerly():\n    tf.compat.v1.disable_eager_execution()","metadata":{"id":"GL3mYAuM_F4G","execution":{"iopub.status.busy":"2022-09-11T21:08:02.178643Z","iopub.execute_input":"2022-09-11T21:08:02.179064Z","iopub.status.idle":"2022-09-11T21:08:07.865347Z","shell.execute_reply.started":"2022-09-11T21:08:02.179022Z","shell.execute_reply":"2022-09-11T21:08:07.864308Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#This code provides helper functions to make output of the network easier to read\n%%capture\n!pip install git+https://github.com/nottombrown/imagenet_stubs\nimport imagenet_stubs\nfrom imagenet_stubs.imagenet_2012_labels import name_to_label, label_to_name","metadata":{"id":"y1s9ysaA_M1O","outputId":"f86fbee7-cad3-4ae0-d88f-308562b7d06a","execution":{"iopub.status.busy":"2022-09-11T21:08:39.301634Z","iopub.status.idle":"2022-09-11T21:08:39.302048Z","shell.execute_reply.started":"2022-09-11T21:08:39.301837Z","shell.execute_reply":"2022-09-11T21:08:39.301863Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#We load the model and prepare utils for preprocessing inputs and outputs\nmodel=tf.keras.applications.MobileNet()\nconvert_output=tf.keras.applications.imagenet_utils.decode_predictions\npreproc=tf.keras.applications.mobilenet.preprocess_input","metadata":{"id":"VNtwGmM9iGfs","outputId":"19d4bf7b-acc2-45a1-f91e-10ae03c465dc","execution":{"iopub.status.busy":"2022-09-11T21:08:21.804356Z","iopub.execute_input":"2022-09-11T21:08:21.805236Z","iopub.status.idle":"2022-09-11T21:08:26.870792Z","shell.execute_reply.started":"2022-09-11T21:08:21.805194Z","shell.execute_reply":"2022-09-11T21:08:26.869661Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#We load the components required for the exploit\nfrom art.utils import to_categorical\nfrom art.estimators.classification import KerasClassifier\nfrom art.attacks.evasion import ProjectedGradientDescent","metadata":{"id":"fJE28U2ph33j","execution":{"iopub.status.busy":"2022-09-11T21:08:26.872468Z","iopub.execute_input":"2022-09-11T21:08:26.873285Z","iopub.status.idle":"2022-09-11T21:08:28.893084Z","shell.execute_reply.started":"2022-09-11T21:08:26.873238Z","shell.execute_reply":"2022-09-11T21:08:28.892043Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#We prepare both the image of chester and of a real hotdog\npixels_dog=np.expand_dims(np.array(Im.open(\"../input/ai-village-ctf/hotterdog/chester.png\").resize((224,224))).astype(\"float32\"),0)\npixels_hotdog=np.expand_dims(np.array(Im.open(\"hotdog_img.jpg\").resize((224,224))).astype(\"float32\"),0)","metadata":{"id":"J1WKddyJiUnw","execution":{"iopub.status.busy":"2022-09-11T21:08:28.894830Z","iopub.execute_input":"2022-09-11T21:08:28.895301Z","iopub.status.idle":"2022-09-11T21:08:28.991747Z","shell.execute_reply.started":"2022-09-11T21:08:28.895251Z","shell.execute_reply":"2022-09-11T21:08:28.990550Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Let's analyze the local and server results","metadata":{}},{"cell_type":"code","source":"img=Im.open(\"../input/ai-village-ctf/hotterdog/chester.png\").resize((224,224))\nimg_bytes = img.tobytes()\nb64_img_bytes = base64.urlsafe_b64encode(img_bytes)\ndata = {\n    \"input\": b64_img_bytes.decode()\n}\nr = requests.post(\"https://hotdog.fly.dev/score\", data=json.dumps(data))\nr.text","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:08:28.994747Z","iopub.execute_input":"2022-09-11T21:08:28.995523Z","iopub.status.idle":"2022-09-11T21:08:29.706505Z","shell.execute_reply.started":"2022-09-11T21:08:28.995457Z","shell.execute_reply":"2022-09-11T21:08:29.705643Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#let's see the output of the network\nconvert_output(model.predict(preproc(pixels_dog.copy())))[0][0]","metadata":{"id":"EgHRILhWikJn","outputId":"c26c72c3-c84b-4394-8ffe-9a55eed894f8","execution":{"iopub.status.busy":"2022-09-11T21:13:33.869370Z","iopub.execute_input":"2022-09-11T21:13:33.869810Z","iopub.status.idle":"2022-09-11T21:13:33.913668Z","shell.execute_reply.started":"2022-09-11T21:13:33.869774Z","shell.execute_reply":"2022-09-11T21:13:33.912835Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We see that both server and local results are \"Rhodesian_ridgeback\" with 0.6210 likelihood (probably the server rounds to 3 digits).","metadata":{}},{"cell_type":"code","source":"#let's see the output of the network for an actual hotdo\nconvert_output(model.predict(preproc(pixels_hotdog.copy())))[0][0]","metadata":{"id":"DPpthgC7jjXn","outputId":"1538018a-a16c-4a52-bf4b-94094b6701ce","execution":{"iopub.status.busy":"2022-09-11T21:13:36.420443Z","iopub.execute_input":"2022-09-11T21:13:36.420830Z","iopub.status.idle":"2022-09-11T21:13:36.463795Z","shell.execute_reply.started":"2022-09-11T21:13:36.420798Z","shell.execute_reply":"2022-09-11T21:13:36.462985Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#get class index for both original class and target class\nbase_class=np.argmax(model.predict(preproc(pixels_dog.copy())))\ntarget_class=np.argmax(model.predict(preproc(pixels_hotdog.copy())))\nprint(f\"image start from class {base_class}. We must change so that it is classified as class {target_class}\")","metadata":{"id":"IdeEE9eW5-hm","outputId":"3e00111f-0ff2-4699-cc8a-6b011a065c0f","execution":{"iopub.status.busy":"2022-09-11T21:08:30.745345Z","iopub.execute_input":"2022-09-11T21:08:30.746016Z","iopub.status.idle":"2022-09-11T21:08:30.819157Z","shell.execute_reply.started":"2022-09-11T21:08:30.745982Z","shell.execute_reply":"2022-09-11T21:08:30.818300Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Prepare the art model","metadata":{}},{"cell_type":"code","source":"#for simplicity we put preoprocessing as a component of art and not inside the network itself\nfrom art.preprocessing.preprocessing import Preprocessor\nclass CustomPreprocessor(Preprocessor):\n\n    def __call__(self, x, y=None):\n        return preproc(x.copy()), y\n\n    def estimate_gradient(self, x, gradient):\n        return gradient[..., ::-1] ","metadata":{"id":"Vf07fcDu9Hg6","execution":{"iopub.status.busy":"2022-09-11T21:08:30.820597Z","iopub.execute_input":"2022-09-11T21:08:30.820892Z","iopub.status.idle":"2022-09-11T21:08:30.826295Z","shell.execute_reply.started":"2022-09-11T21:08:30.820865Z","shell.execute_reply":"2022-09-11T21:08:30.825527Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Visualize the gradient of the image","metadata":{}},{"cell_type":"code","source":"#we create an art classifier to create the adversarial image\nclassifier = KerasClassifier(model=model,  preprocessing=CustomPreprocessor(),clip_values=(0, 255))\nloss_gradient = classifier.loss_gradient(x=pixels_dog, y=to_categorical([target_class], nb_classes=1000))","metadata":{"id":"AUodXKMj6xEd","outputId":"d218c405-fa3a-45a8-ce0b-b97bb2222a6a","execution":{"iopub.status.busy":"2022-09-11T21:08:30.827438Z","iopub.execute_input":"2022-09-11T21:08:30.828229Z","iopub.status.idle":"2022-09-11T21:08:36.169996Z","shell.execute_reply.started":"2022-09-11T21:08:30.828194Z","shell.execute_reply":"2022-09-11T21:08:36.168907Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"loss_gradient_plot = loss_gradient[0] \n# Then normalize loss gradient values to be in [0,1]:\nloss_gradient_min = np.min(loss_gradient)\nloss_gradient_max = np.max(loss_gradient)\nloss_gradient_plot = (loss_gradient_plot - loss_gradient_min)/(loss_gradient_max - loss_gradient_min)\n\n# Show plot:\nplt.figure(figsize=(2,2)); plt.imshow(loss_gradient_plot); plt.axis('off'); plt.show()","metadata":{"id":"zlEN6ChjARTW","outputId":"b6550726-cdf3-4f4e-e0d9-84018dcad77f","execution":{"iopub.status.busy":"2022-09-11T21:08:36.170994Z","iopub.execute_input":"2022-09-11T21:08:36.171307Z","iopub.status.idle":"2022-09-11T21:08:36.320421Z","shell.execute_reply.started":"2022-09-11T21:08:36.171277Z","shell.execute_reply":"2022-09-11T21:08:36.318433Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Let's make Chester an hotdog","metadata":{}},{"cell_type":"code","source":"#instantiate the attack class\nattack = ProjectedGradientDescent(classifier, targeted=True, max_iter=10, eps_step=1, eps=5)\n#generate attack that makes starting image (pixels_dog) become a target_class image\nx_art_adv = attack.generate(pixels_dog,to_categorical([target_class], nb_classes=1000))","metadata":{"id":"StET0L2d7Xhg","execution":{"iopub.status.busy":"2022-09-11T21:08:36.323039Z","iopub.execute_input":"2022-09-11T21:08:36.323680Z","iopub.status.idle":"2022-09-11T21:08:37.341338Z","shell.execute_reply.started":"2022-09-11T21:08:36.323619Z","shell.execute_reply":"2022-09-11T21:08:37.340176Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Visualize the result","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(4,4))\nplt.imshow(x_art_adv[0] / 255)\nplt.show()","metadata":{"id":"-DM41oJfBjJA","outputId":"47790603-780d-4dcd-85b2-8fa8ca9c22b0","execution":{"iopub.status.busy":"2022-09-11T21:08:37.342996Z","iopub.execute_input":"2022-09-11T21:08:37.343345Z","iopub.status.idle":"2022-09-11T21:08:37.539692Z","shell.execute_reply.started":"2022-09-11T21:08:37.343310Z","shell.execute_reply":"2022-09-11T21:08:37.538879Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"convert_output(model.predict(preproc(x_art_adv.astype(\"uint8\").astype(\"float32\").copy())))","metadata":{"id":"JVM9PK8cBvKD","outputId":"9c4dd626-ea7b-4df5-9d59-99406d1ad784","execution":{"iopub.status.busy":"2022-09-11T21:08:37.540889Z","iopub.execute_input":"2022-09-11T21:08:37.541918Z","iopub.status.idle":"2022-09-11T21:08:37.583821Z","shell.execute_reply.started":"2022-09-11T21:08:37.541880Z","shell.execute_reply":"2022-09-11T21:08:37.582557Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Submit the image and get a flag for both hotdog and hotterdog","metadata":{}},{"cell_type":"code","source":"img=Im.fromarray(x_art_adv[0].astype(\"uint8\"))\nimg.save(\"hotterdog.png\")\nimg_bytes = img.tobytes()\n\nb64_img_bytes = base64.urlsafe_b64encode(img_bytes)\n\ndata = {\n    \"input\": b64_img_bytes.decode()\n}\n\nr = requests.post(\"https://hotdog.fly.dev/score\", data=json.dumps(data))\nr.text","metadata":{"id":"5Vzm9OJ07XZz","execution":{"iopub.status.busy":"2022-09-11T21:08:37.585519Z","iopub.execute_input":"2022-09-11T21:08:37.586664Z","iopub.status.idle":"2022-09-11T21:08:38.091140Z","shell.execute_reply.started":"2022-09-11T21:08:37.586618Z","shell.execute_reply":"2022-09-11T21:08:38.089999Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"img_bytes = img.tobytes()\n\nb64_img_bytes = base64.urlsafe_b64encode(img_bytes)\n\ndata = {\n    \"input\": b64_img_bytes.decode()\n}\n\nr = requests.post(\"https://hotterdog.fly.dev/score\", data=json.dumps(data))\nr.text","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:08:38.092756Z","iopub.execute_input":"2022-09-11T21:08:38.093412Z","iopub.status.idle":"2022-09-11T21:08:38.804549Z","shell.execute_reply.started":"2022-09-11T21:08:38.093369Z","shell.execute_reply":"2022-09-11T21:08:38.803524Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Theft \nFrom here the overall structure of the code is the same. \n\nWe are just redoing the attack with different starting image and target class","metadata":{}},{"cell_type":"code","source":"pixels_owl=np.expand_dims(np.array(Im.open(\"../input/ai-village-ctf/theft/owl.jpg\").resize((224,224))).astype(\"float32\"),0)\npixels_turtle=np.expand_dims(np.array(Im.open(\"../input/ai-village-ctf/salt/turtle.jpg\").resize((224,224))).astype(\"float32\"),0)\nbase_class=np.argmax(model.predict(preproc(pixels_owl.copy())))\ntarget_class=np.argmax(model.predict(preproc(pixels_turtle.copy())))\nprint(f\"image start from class {base_class}. We must change so that it is classified as class {target_class}\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:08:38.806000Z","iopub.execute_input":"2022-09-11T21:08:38.806785Z","iopub.status.idle":"2022-09-11T21:08:39.274738Z","shell.execute_reply.started":"2022-09-11T21:08:38.806739Z","shell.execute_reply":"2022-09-11T21:08:39.273464Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"loss_gradient = classifier.loss_gradient(x=pixels_owl, y=to_categorical([target_class], nb_classes=1000))\nloss_gradient_plot = loss_gradient[0] \n\n# Then normalize loss gradient values to be in [0,1]:\nloss_gradient_min = np.min(loss_gradient)\nloss_gradient_max = np.max(loss_gradient)\nloss_gradient_plot = (loss_gradient_plot - loss_gradient_min)/(loss_gradient_max - loss_gradient_min)\n\n# Show plot:\nplt.figure(figsize=(4,4))\nplt.imshow(loss_gradient_plot)\nplt.axis('off')                                         \nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:14:24.809495Z","iopub.execute_input":"2022-09-11T21:14:24.810026Z","iopub.status.idle":"2022-09-11T21:14:25.043411Z","shell.execute_reply.started":"2022-09-11T21:14:24.809977Z","shell.execute_reply":"2022-09-11T21:14:25.041839Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"attack = ProjectedGradientDescent(classifier, targeted=True, max_iter=10, eps_step=1, eps=5)\nx_art_adv = attack.generate(pixels_owl,to_categorical([target_class], nb_classes=1000))\nplt.figure(figsize=(4,4))\nplt.imshow(x_art_adv[0] / 255)\nplt.show()\nimg=Im.fromarray(x_art_adv[0].astype(\"uint8\"))\nimg.save(\"theft-solution-simple.png\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:14:29.369987Z","iopub.execute_input":"2022-09-11T21:14:29.370391Z","iopub.status.idle":"2022-09-11T21:14:30.610053Z","shell.execute_reply.started":"2022-09-11T21:14:29.370357Z","shell.execute_reply":"2022-09-11T21:14:30.608906Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"img_bytes = img.tobytes()\n\nb64_img_bytes = base64.urlsafe_b64encode(img_bytes)\n\ndata = {\n    \"input\": b64_img_bytes.decode()\n}\n\nr = requests.post(\"https://theft.fly.dev/score\", data=json.dumps(data))\nr.text","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:14:31.546830Z","iopub.execute_input":"2022-09-11T21:14:31.547600Z","iopub.status.idle":"2022-09-11T21:14:32.049314Z","shell.execute_reply.started":"2022-09-11T21:14:31.547528Z","shell.execute_reply":"2022-09-11T21:14:32.047887Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Salt\nHere we just reverse the roles of images in Theft","metadata":{}},{"cell_type":"code","source":"base_class=np.argmax(model.predict(preproc(pixels_turtle.copy())))\ntarget_class=np.argmax(model.predict(preproc(pixels_owl.copy())))\nprint(f\"image start from class {base_class}. We must change so that it is classified as class {target_class}\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:14:36.865945Z","iopub.execute_input":"2022-09-11T21:14:36.866316Z","iopub.status.idle":"2022-09-11T21:14:36.973571Z","shell.execute_reply.started":"2022-09-11T21:14:36.866285Z","shell.execute_reply":"2022-09-11T21:14:36.972345Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"loss_gradient = classifier.loss_gradient(x=pixels_turtle, y=to_categorical([target_class], nb_classes=1000))","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:14:37.426032Z","iopub.execute_input":"2022-09-11T21:14:37.426781Z","iopub.status.idle":"2022-09-11T21:14:37.514217Z","shell.execute_reply.started":"2022-09-11T21:14:37.426745Z","shell.execute_reply":"2022-09-11T21:14:37.513211Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"loss_gradient = classifier.loss_gradient(x=pixels_turtle, y=to_categorical([target_class], nb_classes=1000))\nloss_gradient_plot = loss_gradient[0] \n\n# Then normalize loss gradient values to be in [0,1]:\nloss_gradient_min = np.min(loss_gradient)\nloss_gradient_max = np.max(loss_gradient)\nloss_gradient_plot = (loss_gradient_plot - loss_gradient_min)/(loss_gradient_max - loss_gradient_min)\n\n# Show plot:\nplt.figure(figsize=(4,4))\nplt.imshow(loss_gradient_plot)\nplt.axis('off')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:14:40.086945Z","iopub.execute_input":"2022-09-11T21:14:40.087390Z","iopub.status.idle":"2022-09-11T21:14:40.345965Z","shell.execute_reply.started":"2022-09-11T21:14:40.087352Z","shell.execute_reply":"2022-09-11T21:14:40.344923Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"attack = ProjectedGradientDescent(classifier, targeted=True, max_iter=10, eps_step=1, eps=5)\nx_art_adv = attack.generate(pixels_turtle,to_categorical([target_class], nb_classes=1000))\nplt.figure(figsize=(4,4))\nplt.imshow(x_art_adv[0] / 255)\nplt.show()\nimg=Im.fromarray(x_art_adv[0].astype(\"uint8\"))\nimg.save(\"salt-solution-simple.png\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:14:41.086688Z","iopub.execute_input":"2022-09-11T21:14:41.087780Z","iopub.status.idle":"2022-09-11T21:14:42.911724Z","shell.execute_reply.started":"2022-09-11T21:14:41.087729Z","shell.execute_reply":"2022-09-11T21:14:42.910490Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"img_bytes = img.tobytes()\n\nb64_img_bytes = base64.urlsafe_b64encode(img_bytes)\n\ndata = {\n    \"input\": b64_img_bytes.decode()\n}\n\nr = requests.post(\"https://salt.fly.dev/score\", data=json.dumps(data))\nr.text","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:14:42.913630Z","iopub.execute_input":"2022-09-11T21:14:42.913971Z","iopub.status.idle":"2022-09-11T21:14:43.582836Z","shell.execute_reply.started":"2022-09-11T21:14:42.913937Z","shell.execute_reply":"2022-09-11T21:14:43.581467Z"},"trusted":true},"execution_count":null,"outputs":[]}]}