{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Solutions Notebook\n\nThis notebook contains my final solutions to the challenges I solved during the competition. It does not contain the many failed attempts at some of the challenges before arriving at a solution, but I will comment on some of those throughout.","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport os\nimport requests\nfrom IPython.display import Image as Ima\nfrom IPython.display import clear_output, display\nfrom PIL import Image as Im\nfrom torch.utils.data import Dataset, DataLoader\nimport torch\nimport joblib\nimport matplotlib.pyplot as plt\nimport json\nimport base64\nimport shutil\nfrom copy import deepcopy\nimport warnings\nwarnings.filterwarnings(\"ignore\")\n\n\ndir_name = \"../input/\"","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:24:01.244767Z","iopub.execute_input":"2022-09-15T18:24:01.245704Z","iopub.status.idle":"2022-09-15T18:24:01.903763Z","shell.execute_reply.started":"2022-09-15T18:24:01.245590Z","shell.execute_reply":"2022-09-15T18:24:01.902358Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# HOTDOG (id: hotdog) -- 200 Points\n\n### Scenario\n\nThis is Chester. He's heard of Hotdog/Not Hotdog, but really wants to be classified as a hot-dog (get it?).\n\n### Objective\n\nHelp Chester classify as a hotdog.\n\n### Solution\n\nI started simple, found a picture of a hotdog and submitted it with success!","metadata":{}},{"cell_type":"code","source":"# Submission Helper\n\nIMAGE_DIMS = (224, 224)\n\nimg = Im.open(f\"{dir_name}/hotdog/hotdog.png\")\nimg = img.resize(IMAGE_DIMS)\nimg_bytes = img.tobytes()\nb64_img_bytes = base64.urlsafe_b64encode(img_bytes)\n\ntry:\n    r = requests.post(\"https://hotdog.fly.dev/score\", data=json.dumps({\"input\": b64_img_bytes.decode()}))\n    print(r.text)\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")\nimg","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:24:04.006666Z","iopub.execute_input":"2022-09-15T18:24:04.007280Z","iopub.status.idle":"2022-09-15T18:24:04.412868Z","shell.execute_reply.started":"2022-09-15T18:24:04.007243Z","shell.execute_reply":"2022-09-15T18:24:04.411296Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Math\n\nThis series of \"Math\" challenges is designed to introduce the concepts of dimensionality.\n\nData is often represented in a high dimensional space, but is actually several intrinsically low dimensional components that can be diagnosed. Intrinsic low-dimensionailty is incredibly important to machine learning because many ML algorithms rely on it. Medical EKG data is as single intrinsically 2 dimensional component, despite often being represented as 100 dimensions or more. The classic MNIST dataset is several components of 8-10 dimensions, depending on the digits class, despite being represented in 784 dimensions. The 4 math challenges are focused on this problem. **If you need a hint, ask Sven at the Art Booth.**","metadata":{}},{"cell_type":"markdown","source":"# Math Challenge 1 (id: math_1) -- 100 Points\n\nHow many clusters are in the clusters1.npy, clusters2.npy, and clusters3.npy? The key is the number of clusters in order, with no spaces. These files are available in the input directory under `math`.\n\n### Solution:\nI iterate over each file and cluster with KMeans with the number of clusters varying from 2 to 9. I choose the number of clusters that achieve the best silhouette score.","metadata":{}},{"cell_type":"code","source":"from sklearn.cluster import KMeans\nfrom sklearn.metrics import silhouette_score","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:24:07.278585Z","iopub.execute_input":"2022-09-15T18:24:07.279040Z","iopub.status.idle":"2022-09-15T18:24:07.726135Z","shell.execute_reply.started":"2022-09-15T18:24:07.279002Z","shell.execute_reply":"2022-09-15T18:24:07.724881Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for j in [1,2,3]:\n    X = np.load(dir_name + f'ai-village-ctf/math/clusters{j}.npy')\n    best_score = 0\n    n_clusters = -1\n    for i in range(2, 10):\n        model = KMeans(n_clusters=i)\n        model.fit(X)\n        score = silhouette_score(X, model.labels_)\n        if score > best_score:\n            best_score = score\n            n_clusters = i\n    print(f'num_clusters: {n_clusters}')","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:24:09.351433Z","iopub.execute_input":"2022-09-15T18:24:09.351890Z","iopub.status.idle":"2022-09-15T18:24:10.178931Z","shell.execute_reply.started":"2022-09-15T18:24:09.351842Z","shell.execute_reply":"2022-09-15T18:24:10.176187Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\ntry:\n    r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"math_1\", \"submission\": \"523\"}))\n    print(r.text)\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:24:10.607052Z","iopub.execute_input":"2022-09-15T18:24:10.607508Z","iopub.status.idle":"2022-09-15T18:24:10.824566Z","shell.execute_reply.started":"2022-09-15T18:24:10.607470Z","shell.execute_reply":"2022-09-15T18:24:10.823508Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Math Challenge 2 (id: math_2) -- 200 Points\n\nWhat's the dimensionality of the data in first_dim1.npy, first_dim2.npy, and first_dim3.npy? The key is the number of dimensions in order, with no spaces.  These files are available in the input directory under `math`.\n\n### Solution\nKMeans has an assumption that the clusters are spherical, or at least have linear decision boundaries. The data here didn't quite fit the bill. This time I go after the approximate rank of the matrix. The rank of a matrix is the number of non-zero singular values and is a representation of the \"dimensionality\" of the underlying data. ","metadata":{}},{"cell_type":"code","source":"for j in [1,2,3]:\n    X = np.load(dir_name + f'ai-village-ctf/math/first_dim{j}.npy')\n    _, s, _ = np.linalg.svd(X)\n    print((s > 1e-8).sum())","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:24:15.903169Z","iopub.execute_input":"2022-09-15T18:24:15.904478Z","iopub.status.idle":"2022-09-15T18:24:15.919112Z","shell.execute_reply.started":"2022-09-15T18:24:15.904385Z","shell.execute_reply":"2022-09-15T18:24:15.918139Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\ntry:\n    r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"math_2\", \"submission\": \"354\"}))\n    print(r.text)\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:24:16.939880Z","iopub.execute_input":"2022-09-15T18:24:16.941304Z","iopub.status.idle":"2022-09-15T18:24:17.153542Z","shell.execute_reply.started":"2022-09-15T18:24:16.941235Z","shell.execute_reply":"2022-09-15T18:24:17.152512Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Math Challenge 3 (id: math_3) -- 300 Points\n\nWhat's the dimensionality of the data in second_dim1.npy, second_dim2.npy, and second_dim3.npy? The key is the number of clusters in order, with no spaces.  These files are available in the input directory under `math`.\n\n### Solution\nThis data isn't as clean as the above, so a true principal components analysis was required. It's basically the same technique, except the data needs to be centered and scaled to unit variance per dimension. ","metadata":{}},{"cell_type":"code","source":"for j in [1,2,3]:\n    X = np.load(dir_name + f'ai-village-ctf/math/second_dim{j}.npy')\n    X -= X.mean(0)\n    X /= X.std(0)\n    _, s, _ = np.linalg.svd(X)\n    print((s > 1e-8).sum())","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:24:20.600220Z","iopub.execute_input":"2022-09-15T18:24:20.600803Z","iopub.status.idle":"2022-09-15T18:24:20.614090Z","shell.execute_reply.started":"2022-09-15T18:24:20.600756Z","shell.execute_reply":"2022-09-15T18:24:20.612946Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\ntry:\n    r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"math_3\", \"submission\": \"474\"}))\n    print(r.text)\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:24:22.173342Z","iopub.execute_input":"2022-09-15T18:24:22.174042Z","iopub.status.idle":"2022-09-15T18:24:22.393276Z","shell.execute_reply.started":"2022-09-15T18:24:22.174003Z","shell.execute_reply":"2022-09-15T18:24:22.391493Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Math Challenge 4 (id: math_4) -- 400 Points\n\nWhat's the dimensionality of the clusters in clusters1.npy? The key is the dimensions ordered by cluster size (smallest to largest), with no spaces.  These files are available in the input directory under `math`.\n\n### Solution:\nThis is the same file as in math 1, so I attack it with KMeans with the optimal number of clusters found before. The trick here was that the author wanted the dimension of each cluster ordered by cluster size.\n\n1. Cluster the data\n2. Order clusters by size\n3. For each cluster find the approximate rank as done in challenge 2.","metadata":{}},{"cell_type":"code","source":"result = ''\nX = np.load(dir_name + f'ai-village-ctf/math/clusters1.npy')\nmodel = KMeans(n_clusters=5)\nmodel.fit(X)\nlabels = model.predict(X)\nclusters, counts = np.unique(labels, return_counts = True)\nclusters = clusters[np.argsort(counts)]\nfor cluster in clusters:\n    Y = X[np.where(labels == cluster)[0]]\n    _, s, _ = np.linalg.svd(X[np.where(labels == cluster)[0]])\n    result += str((s > 1e-8).sum() - 1)\n\n# Submission Helper\ntry:\n    r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"math_4\", \"submission\": result}))\n    print(r.text)\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:24:24.549575Z","iopub.execute_input":"2022-09-15T18:24:24.550123Z","iopub.status.idle":"2022-09-15T18:24:24.847040Z","shell.execute_reply.started":"2022-09-15T18:24:24.550078Z","shell.execute_reply":"2022-09-15T18:24:24.845546Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# WIFI (id: wifi) -- 200 Points\n\n### Scenario\n\nYou really need to check your email, unfortunately you don't know the password.  Fortunately, someone wrote it down.  Unfortunately, it's written down on a low-dimensional manifold embedded in a very high-dimensional space.  Check out the `wifi/Embedded characters.npz` file -- a list of tokens is given in the `tokens` key with their corresponding embeddings in the same order under the `embeddings` key -- and recover the password.\n\n### Objective\n\nThe password is your flag, like all others, it has the format `FLAG{something}`\n\n### Solution\n\nI used Multidimensional Scaling to extract and plot the lower dimensional manifold. To extract the tokens in order I used each points distance from the origin. The flag was the first part of the tokens.","metadata":{}},{"cell_type":"code","source":"with np.load(dir_name + f'ai-village-ctf/wifi/Embedded_characters.npz') as data:\n    tokens = data['tokens']\n    embeddings = data['embeddings']","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:24:27.323085Z","iopub.execute_input":"2022-09-15T18:24:27.323680Z","iopub.status.idle":"2022-09-15T18:24:27.338361Z","shell.execute_reply.started":"2022-09-15T18:24:27.323612Z","shell.execute_reply":"2022-09-15T18:24:27.337389Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.manifold import MDS\n\nlow = MDS().fit_transform(embeddings)\nplt.scatter(low[:, 0], low[:, 1]);","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:24:29.012370Z","iopub.execute_input":"2022-09-15T18:24:29.013820Z","iopub.status.idle":"2022-09-15T18:24:29.753526Z","shell.execute_reply.started":"2022-09-15T18:24:29.013758Z","shell.execute_reply":"2022-09-15T18:24:29.751845Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.metrics.pairwise import euclidean_distances\nd = euclidean_distances(low, [[0,0]]).ravel()\n''.join([tokens.tolist()[i] for i in d.argsort()])","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:24:30.804141Z","iopub.execute_input":"2022-09-15T18:24:30.805280Z","iopub.status.idle":"2022-09-15T18:24:30.816058Z","shell.execute_reply.started":"2022-09-15T18:24:30.805226Z","shell.execute_reply":"2022-09-15T18:24:30.814669Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\ntry:\n    r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"wifi\", \"submission\": \"FLAG{TURNED}\"}))\n    print(r.text)\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:24:32.515164Z","iopub.execute_input":"2022-09-15T18:24:32.516851Z","iopub.status.idle":"2022-09-15T18:24:32.736111Z","shell.execute_reply.started":"2022-09-15T18:24:32.516771Z","shell.execute_reply":"2022-09-15T18:24:32.734610Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# HOTTERDOG (id: hotterdog) -- 400 Points\n\n### Scenario\n\nChester submitted himself in the \"most creative hotdog\" contest. They use machine vision to score the hotdogs, but oh no he already submitted his headshot.\n\n### Objective\n\nHelp Chester classify as a hotdog, but it has to look like Chester to a human. Modify his picture to fool a classifier.\n\n### Solution\n\nThis time the image couldn't be manipulated past some threshold. I combined the hotdog image from before with the original image, and rotated the hotdog such that it aligned with the actual dog to reduce the modification penalty. ","metadata":{}},{"cell_type":"code","source":"# Submission Helper\n\nIMAGE_DIMS = (224, 224)\n\nimg = Im.open(f\"{dir_name}ai-village-ctf/hotdog/chester.png\")\nimg = img.resize(IMAGE_DIMS)\nimg","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:24:34.206533Z","iopub.execute_input":"2022-09-15T18:24:34.206979Z","iopub.status.idle":"2022-09-15T18:24:34.310484Z","shell.execute_reply.started":"2022-09-15T18:24:34.206945Z","shell.execute_reply":"2022-09-15T18:24:34.308602Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"hd = Im.open(f\"{dir_name}hotterdog/hotdog.jpg\").rotate(90)\nhd = hd.resize(IMAGE_DIMS)\nhd","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:24:35.610083Z","iopub.execute_input":"2022-09-15T18:24:35.611985Z","iopub.status.idle":"2022-09-15T18:24:35.654887Z","shell.execute_reply.started":"2022-09-15T18:24:35.611926Z","shell.execute_reply":"2022-09-15T18:24:35.653628Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"img_array = np.array(img).astype(float)\nhd_array = np.array(hd).astype(float)\nimg_array = (0.59 * img_array + 0.41 * hd_array).astype(np.uint8)\nattack_img = Im.fromarray(img_array)\nattack_img","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:24:37.131715Z","iopub.execute_input":"2022-09-15T18:24:37.133087Z","iopub.status.idle":"2022-09-15T18:24:37.179260Z","shell.execute_reply.started":"2022-09-15T18:24:37.133025Z","shell.execute_reply":"2022-09-15T18:24:37.177521Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"img_bytes = attack_img.tobytes()\nb64_img_bytes = base64.urlsafe_b64encode(img_bytes)\n\ntry:\n    r = requests.post(\"https://hotterdog.fly.dev/score\", data=json.dumps({\"input\": b64_img_bytes.decode()}))\n    print(r.text)\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:24:38.532376Z","iopub.execute_input":"2022-09-15T18:24:38.532861Z","iopub.status.idle":"2022-09-15T18:24:38.848901Z","shell.execute_reply.started":"2022-09-15T18:24:38.532824Z","shell.execute_reply":"2022-09-15T18:24:38.847457Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Honorstudent (id: honorstudent) -- 300 Points\n\n### Scenario\n\nLook at your report card! That's an F. You won't be able to go to homecoming with that grade. Can you use some carefully placed whiteout to fool your parents into thinking it's an A?\n\n### Objective\n\nModify and submit your tampered `f.png`. Be careful to not overdo it!\n\n### Solution\n\nThe emnist dataset contains images of letters and numbers that are the correct size for this challenge. I grab the first image of an 'A' and submit.","metadata":{}},{"cell_type":"code","source":"%%capture\n!pip install emnist","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:24:41.005090Z","iopub.execute_input":"2022-09-15T18:24:41.005593Z","iopub.status.idle":"2022-09-15T18:24:52.369291Z","shell.execute_reply.started":"2022-09-15T18:24:41.005556Z","shell.execute_reply":"2022-09-15T18:24:52.367438Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from emnist import extract_training_samples\nimages, labels = extract_training_samples('letters')","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:24:52.371703Z","iopub.execute_input":"2022-09-15T18:24:52.372326Z","iopub.status.idle":"2022-09-15T18:24:53.340289Z","shell.execute_reply.started":"2022-09-15T18:24:52.372284Z","shell.execute_reply":"2022-09-15T18:24:53.339147Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.imshow(images[labels==1][0]);","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:24:53.341779Z","iopub.execute_input":"2022-09-15T18:24:53.342257Z","iopub.status.idle":"2022-09-15T18:24:53.574064Z","shell.execute_reply.started":"2022-09-15T18:24:53.342212Z","shell.execute_reply":"2022-09-15T18:24:53.572614Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"A = Im.fromarray(images[labels==1][0])\nA.save(f'{dir_name}honorstudent/A.png')","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:24:56.293185Z","iopub.execute_input":"2022-09-15T18:24:56.293733Z","iopub.status.idle":"2022-09-15T18:24:56.303004Z","shell.execute_reply.started":"2022-09-15T18:24:56.293687Z","shell.execute_reply":"2022-09-15T18:24:56.301660Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\nwith open(f\"{dir_name}honorstudent/A.png\", \"rb\") as f:\n    try:\n        r = requests.post(\"https://honorstudent.fly.dev/score\", files={\"data_file\": f})\n        print(r.text)\n    except requests.exceptions.ConnectionError:\n        print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:25:02.312918Z","iopub.execute_input":"2022-09-15T18:25:02.313343Z","iopub.status.idle":"2022-09-15T18:25:02.569007Z","shell.execute_reply.started":"2022-09-15T18:25:02.313311Z","shell.execute_reply":"2022-09-15T18:25:02.567513Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Secret Sloth (id: sloth) -- 400 Points\n\n### Scenario\n\nThe sloth in `secret.sloth/secret-sloth.png` has a very cool hat... and a secret message for you. The message is your flag.\n\n### Objective \n\nThe flag is in the image somewhere; as always, look for `FLAG{something}`\n\n### Solution\n\nI tried a few steg techniques such as least significant bit attacks, before finding the original image online. When comparing the difference between this image and the original I could see where the modifications were located. The modifications were too large to be the traditional type of bit attacks. When looking at the image of the difference between the two images I noticed artifacts around the borders. I had seen similar artifacts in a Computer Vision course when doing FFTs of images. The flag was cleverly placed in the real component of the FFT.","metadata":{}},{"cell_type":"code","source":"img = Im.open(f'{dir_name}ai-village-ctf/secret.sloth/secret-sloth.png')\nimg","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:25:06.567276Z","iopub.execute_input":"2022-09-15T18:25:06.567810Z","iopub.status.idle":"2022-09-15T18:25:06.722043Z","shell.execute_reply.started":"2022-09-15T18:25:06.567772Z","shell.execute_reply":"2022-09-15T18:25:06.720763Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"f = np.fft.fft2(np.array(img)/255)\nplt.imshow(f.real);","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:25:07.679818Z","iopub.execute_input":"2022-09-15T18:25:07.680313Z","iopub.status.idle":"2022-09-15T18:25:08.007549Z","shell.execute_reply.started":"2022-09-15T18:25:07.680276Z","shell.execute_reply":"2022-09-15T18:25:08.006105Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\ntry:\n    r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"sloth\", \"submission\": \"FLAG{SPECTRAL}\"}))\n    print(r.text)\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:25:09.228190Z","iopub.execute_input":"2022-09-15T18:25:09.228962Z","iopub.status.idle":"2022-09-15T18:25:09.442463Z","shell.execute_reply.started":"2022-09-15T18:25:09.228909Z","shell.execute_reply":"2022-09-15T18:25:09.441512Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Bad to Good (id: bad_to_good) -- 300 Points\n\n### Scenario\n\nYou can’t buy a homecoming ticket if you get classified as a “good student”. Can you help Henry poison the classifier? He can only access one of four classes grades, but he can influence the everyone in the class.\n\n### Objective\n\nModify and submit your tampered `csv`. Try and get Henry classified as a good student. Be careful to not overdo it!\n\n### Solution\n\nNothing too clever here, I just modifications to the file until it worked.","metadata":{}},{"cell_type":"code","source":"df = pd.read_csv(f\"{dir_name}ai-village-ctf/bad2good/math.csv\")\ndf.loc[df.FirstName == \"Henry\", 'Demerits'] = -1\ndf.loc[df.FirstName != \"Henry\", 'Demerits'] = 9\ndf.loc[df.FirstName == \"Henry\", 'Absences'] = 0\ndf.loc[df.FirstName != \"Henry\", 'Absences'] = 9\ndf.loc[df.FirstName == \"Henry\", 'Grade'] = 210\ndf.loc[df.FirstName != \"Henry\", 'Grade'] = 0\ndf.loc[df.FirstName == \"Henry\", 'Score'] = 100\ndf.loc[df.FirstName != \"Henry\", 'Score'] = 0\ndf.to_csv(f\"{dir_name}bad2good/mod.csv\")","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:25:15.033562Z","iopub.execute_input":"2022-09-15T18:25:15.034998Z","iopub.status.idle":"2022-09-15T18:25:15.069054Z","shell.execute_reply.started":"2022-09-15T18:25:15.034909Z","shell.execute_reply":"2022-09-15T18:25:15.066331Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\nwith open(f\"{dir_name}bad2good/mod.csv\", \"r\") as f:\n    try:\n        r = requests.post(\"https://bad2good.fly.dev/score\", files={\"data_file\": f})\n        print(r.text)\n    except requests.exceptions.ConnectionError:\n        print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:25:18.322656Z","iopub.execute_input":"2022-09-15T18:25:18.323198Z","iopub.status.idle":"2022-09-15T18:25:18.571963Z","shell.execute_reply.started":"2022-09-15T18:25:18.323160Z","shell.execute_reply":"2022-09-15T18:25:18.570895Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Inference (id: inference) -- 400 Points\n\n### Scenario\n\nModel model on the wall, who is the fairest of them all?\n\n?gnidaer llits uoy era yhW .uoy htiw skcans thguorb ev'dluohs uoY .skcans rof pots t'noD .won oG .tsaf oG .niW .galf eht teG .ledom eht esreveR\n\n### Objective\n\nRetrieve the images that spell the flag. Wrap any text in `FLAG{}`\n\n**If you need a hint, there's a guy named `moo` somewhere.**\n\n### Solution\n\nMy first attempt was to start with random noise and do a random hill climbing algorithm to maximize the output for each class. The only image that I could see clearly after the approach was an image of a '3' at location 2. \n\nThen I pulled out the emnist dataset again and pushed the first 40 images of each capital letter and digit through the model. I recorded the output class and if the model was confident (greater than 90% sure). I looked at which characters the model was most confident for each position and saw that D3FC0N was an option.","metadata":{}},{"cell_type":"code","source":"from emnist import extract_training_samples\nimages, labels = extract_training_samples('byclass')\nimages = images / 255.","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:25:24.697552Z","iopub.execute_input":"2022-09-15T18:25:24.698799Z","iopub.status.idle":"2022-09-15T18:25:41.448742Z","shell.execute_reply.started":"2022-09-15T18:25:24.698758Z","shell.execute_reply":"2022-09-15T18:25:41.447439Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def submit(x):\n    try:\n        response = requests.post(\"https://inference.fly.dev/score\", data = json.dumps({\"input\": x.tolist()}))\n        ret = np.array(json.loads(response.text)[\"outputs\"]).ravel()\n        c = ret.argmax()\n        conf = ret[c] > .9\n        return c, conf\n    except requests.exceptions.ConnectionError:\n        print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:25:41.451369Z","iopub.execute_input":"2022-09-15T18:25:41.451889Z","iopub.status.idle":"2022-09-15T18:25:41.459982Z","shell.execute_reply.started":"2022-09-15T18:25:41.451841Z","shell.execute_reply":"2022-09-15T18:25:41.458499Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from tqdm import trange\n\ncounts = np.zeros((36, 6))\nconfs = np.zeros((36, 6))\nfor label in trange(36):\n    idx = np.where(labels==label)[0]\n    np.random.shuffle(idx)\n    for img in images[idx[:40]]:\n        x = np.array(Im.fromarray(img).resize((32, 32))).reshape((1, 32, 32, 1))\n        c, conf = submit(x)\n        counts[label, c] += 1\n        if conf:\n            confs[label, c] += 1","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:25:41.461423Z","iopub.execute_input":"2022-09-15T18:25:41.461855Z","iopub.status.idle":"2022-09-15T18:30:37.267349Z","shell.execute_reply.started":"2022-09-15T18:25:41.461820Z","shell.execute_reply":"2022-09-15T18:30:37.265863Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import string\nclasses = string.digits + string.ascii_uppercase","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:30:37.269897Z","iopub.execute_input":"2022-09-15T18:30:37.270275Z","iopub.status.idle":"2022-09-15T18:30:37.276864Z","shell.execute_reply.started":"2022-09-15T18:30:37.270240Z","shell.execute_reply":"2022-09-15T18:30:37.275292Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"letters = []\nscores = []\nfor c in confs.T:\n    letters.append([classes[i] for i in np.argsort(c)[::-1][:10]])\n    scores.append(c[np.argsort(c)[::-1]][:10])\nnp.array(letters), np.array(scores)","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:30:37.278617Z","iopub.execute_input":"2022-09-15T18:30:37.279341Z","iopub.status.idle":"2022-09-15T18:30:37.297677Z","shell.execute_reply.started":"2022-09-15T18:30:37.279304Z","shell.execute_reply":"2022-09-15T18:30:37.296331Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# To try and get the flag\ntry:\n    r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"inference\", \"submission\": \"FLAG{D3FC0N}\"}))\n    print(r.text)\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:30:37.299786Z","iopub.execute_input":"2022-09-15T18:30:37.300269Z","iopub.status.idle":"2022-09-15T18:30:37.512069Z","shell.execute_reply.started":"2022-09-15T18:30:37.300224Z","shell.execute_reply":"2022-09-15T18:30:37.510393Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Baseball  (id: baseball) -- 300 Points\n\n### Scenario\n\nHenry has to miss baseball tryouts to get ready for his date. Can you cover for him? Henry is a pitcher and the team uses software to analyze pitch patterns to classify pitchers. \n\n### Objective\n\nThrow 15 pitches at x,y (each bounded on [0,29]). Check out `test_pitch.json` to see the expected format. Can you throw 15 pitches to get classified as Henry?\n\n### Solutions\n\nI started with pitches in the center of the range and randomly swapped out pitches and saved the swap if it improved my score.","metadata":{}},{"cell_type":"code","source":"with open(f\"{dir_name}ai-village-ctf/baseball/test_pitch.json\", \"r\") as f:\n    test = json.load(f)\npitches = []\nfor x in range(10, 15, 2):\n    for y in range(9, 14):\n        pitches.append({'x':str(x), 'y':str(y)})\ntest['pitches'] = pitches","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:39:46.361057Z","iopub.execute_input":"2022-09-15T18:39:46.364620Z","iopub.status.idle":"2022-09-15T18:39:46.383301Z","shell.execute_reply.started":"2022-09-15T18:39:46.364528Z","shell.execute_reply":"2022-09-15T18:39:46.381740Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\ndef submit_baseball(test):\n    try:\n        r = requests.post(\"https://bball.fly.dev/score\", json=test)\n        return float(r.text.split(':')[-1].split('\"')[0][:-1]), r.text\n    except requests.exceptions.ConnectionError:\n        print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:39:48.005163Z","iopub.execute_input":"2022-09-15T18:39:48.005944Z","iopub.status.idle":"2022-09-15T18:39:48.014280Z","shell.execute_reply.started":"2022-09-15T18:39:48.005900Z","shell.execute_reply":"2022-09-15T18:39:48.012530Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"np.random.seed(0)\nscore, text = submit_baseball(test)\nfor _ in range(50):\n    pitch = np.random.randint(0, 30), np.random.randint(0, 30)\n    temp = deepcopy(test)\n    temp['pitches'][np.random.randint(0, 15)] = {'x' : str(pitch[0]), 'y' : str(pitch[1])}\n    try:\n        s, text = submit_baseball(temp)\n        if s > score:\n            score = s\n            test = temp\n            print(text)\n    except:\n        r = requests.post(\"https://bball.fly.dev/score\", json=temp)\n        print(r.text)","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:39:50.680757Z","iopub.execute_input":"2022-09-15T18:39:50.681946Z","iopub.status.idle":"2022-09-15T18:40:10.244117Z","shell.execute_reply.started":"2022-09-15T18:39:50.681899Z","shell.execute_reply":"2022-09-15T18:40:10.242973Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# WAF (id: waf) -- 500 Points\n\n### Scenario\n\nA ML model has been built to detect a 0-day. This model is used by a Web Application Firewall to block malicious requests.\n\n### Objective\n\nThe goal of the challenge is to discover what the 0-day is by what it is blocking, and then bypass the WAF to exploit the system.\n\nPS: The WAF assesses malicious strings in 5 characters blocks.\n\n### Solution\n\nThe model checks groups of 5 characters and we were given a starting point that appears to be the end of a base64 encoded string. I took the first 4 characters and tested every valid base64 character to see if I could produce another malicious chunk. Once I found the character I repeated the process until I had a recognizable string (shell shock exploit). To trick the model I inserted the base64 padding character '=' to break up the malicious chunks of 5.","metadata":{}},{"cell_type":"code","source":"s = \"mFzaC==\"\nsolved = False\nwhile not solved:\n    solved = True\n    for c in string.printable:\n        input = f\"{c}{s[:4]}\"\n        data = json.dumps(\n            {\"input\": input}\n        )\n\n        r = requests.post(\"https://waf.fly.dev/score\", data=data)\n        if r.text.split('\"')[1] == \"error\":\n            continue\n        elif r.text.split('\"')[3] == \"REQUEST IS SAFE\":\n            continue\n        else:\n            solved = False\n            s = c + s\n            try:\n                print(base64.b64decode(s.encode()).decode())\n                print(s)\n            except:\n                pass\n            break","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:40:15.200352Z","iopub.execute_input":"2022-09-15T18:40:15.201510Z","iopub.status.idle":"2022-09-15T18:45:59.773806Z","shell.execute_reply.started":"2022-09-15T18:40:15.201457Z","shell.execute_reply":"2022-09-15T18:45:59.772185Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#shell shock isn't a zero day :(\ninput = \"KCkg=eyA6=O307=IC9i=aW4v=YmFz=aC==\"\ndata = json.dumps(\n    {\"input\": input}\n)\n\ntry:\n    r = requests.post(\"https://waf.fly.dev/score\", data=data)\n    print(r.text)\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:45:59.777252Z","iopub.execute_input":"2022-09-15T18:45:59.777826Z","iopub.status.idle":"2022-09-15T18:46:00.185810Z","shell.execute_reply.started":"2022-09-15T18:45:59.777772Z","shell.execute_reply":"2022-09-15T18:46:00.184378Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Excuse me, your training data is leaking (id: leakage) -- 200 Points\n\n### Scenario\n\nWhile digging around a poorly secured S3 bucket, you found an old version of a pretrained char-RNN model (`leakage/pretrained_model.pt`). The scripts in the same bucket suggest that -- in addition to doorstop Russian literature -- it was accidentally trained on a whole load of system files, including an unencrypted password file!  Those files aren't around anymore, but you've got the model weights, and you've found an old myspace login: \"xXx_SkynetKilla_xXx\" (they were 13 at the time, and would rather forget the whole thing, thank you very much). Can you recover their password?\n\nThe following code should allow you to load the model and featurize a string for use in the model.\n```python\nimport torch\nfrom torch import nn\nfrom torch.nn import functional as F\n\nclass LM(nn.Module):\n    def __init__(self):\n        super(LM, self).__init__()\n        self.E = nn.Embedding(256, 32)\n        self.LSTM = nn.LSTM(32,128, 2, batch_first=True, dropout=.1)\n        self.predict = nn.Sequential(\n            nn.Linear(128, 128),\n            nn.LayerNorm(128),\n            nn.ELU(),\n            nn.Linear(128, 256)\n        )\n    def forward(self, x):\n        h = self.E(x)\n        h = self.LSTM(h)[0][:,-1,:]\n        h = self.predict(h)\n        return h\n\ndef featurize(s):\n    return torch.Tensor([[int(ord(x)) for x in s]]).long()\n\nmodel = LM()\nmodel.load_state_dict(torch.load(\"pretrained_model.pt\"))\n```\n\n\n### Objective\n\nThe password is your flag, as usual it is in the format `FLAG{something}`\n\n### Solution\n\nI instantiated the model and started with the given user name to produce the next most likely character. I repeated this process until I grew the string passed the FLAG.\n","metadata":{}},{"cell_type":"code","source":"import torch\nfrom torch import nn\nfrom torch.nn import functional as F\n\nclass LM(nn.Module):\n    def __init__(self):\n        super(LM, self).__init__()\n        self.E = nn.Embedding(256, 32)\n        self.LSTM = nn.LSTM(32, 128, 2, batch_first=True, dropout = .1)\n        self.predict = nn.Sequential(\n            nn.Linear(128, 128),\n            nn.LayerNorm(128),\n            nn.ELU(),\n            nn.Linear(128, 256))\n        \n    def forward(self, x):\n        h = self.E(x)\n        h = self.LSTM(h)[0][:, -1, :]\n        h = self.predict(h)\n        return h\n    \ndef featureize(s):\n    return torch.Tensor([[int(ord(x)) for x in s]]).long()\n\ndef tokenize(vec):\n    vec\n\nmodel = LM()\nmodel.load_state_dict(torch.load(f'{dir_name}ai-village-ctf/leakage/pretrained_model.pt'))","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:51:40.654718Z","iopub.execute_input":"2022-09-15T18:51:40.655254Z","iopub.status.idle":"2022-09-15T18:51:40.692047Z","shell.execute_reply.started":"2022-09-15T18:51:40.655201Z","shell.execute_reply":"2022-09-15T18:51:40.691019Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"s = \"xXx_SkynetKilla_xXx\"\nwhile s[-1] != '}':\n    s += chr(model(featureize(s)).argmax())\nprint(s)","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:51:42.487136Z","iopub.execute_input":"2022-09-15T18:51:42.489102Z","iopub.status.idle":"2022-09-15T18:51:42.643277Z","shell.execute_reply.started":"2022-09-15T18:51:42.489033Z","shell.execute_reply":"2022-09-15T18:51:42.641616Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\ntry:\n    r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"leakage\", \"submission\": \"FLAG{s4RaHc0Nnor<3<3}\"}))\n    print(r.text)\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:51:45.159767Z","iopub.execute_input":"2022-09-15T18:51:45.160725Z","iopub.status.idle":"2022-09-15T18:51:45.383549Z","shell.execute_reply.started":"2022-09-15T18:51:45.160671Z","shell.execute_reply":"2022-09-15T18:51:45.382105Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# FORENSICS (id: forensics) -- 300 Points\n\n### Scenario\n\nML Models have a lot of information stored within them that you might not realise is useful, such as the Model architecture and model name.\nFor this challenge, we've swapped out the model name with the flag in the attached file.\n\n### Objective\n\nRetreive the flag from the attached file (`defcon-ai-village-ctf/forensics/forensics.h5`). Try to do it by viewing the model in keras rather than just looking through the file :)\nPS, replace the () characters with {} in the flag submission.\n\n### Solution\n\nPrinting out the (very verbose) model summary turned out to be all that was required to find the flag.","metadata":{}},{"cell_type":"code","source":"import keras\nmodel = keras.models.load_model(f'{dir_name}ai-village-ctf/forensics/forensics.h5')\nmodel.summary()","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:51:49.638937Z","iopub.execute_input":"2022-09-15T18:51:49.639553Z","iopub.status.idle":"2022-09-15T18:51:53.725039Z","shell.execute_reply.started":"2022-09-15T18:51:49.639504Z","shell.execute_reply":"2022-09-15T18:51:53.721993Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\ntry:\n    r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"forensics\", \"submission\": \"FLAG{F0R3NS1C5_XD}\"}))\n    print(r.text)\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:51:53.728081Z","iopub.execute_input":"2022-09-15T18:51:53.729591Z","iopub.status.idle":"2022-09-15T18:51:53.944977Z","shell.execute_reply.started":"2022-09-15T18:51:53.729532Z","shell.execute_reply":"2022-09-15T18:51:53.943794Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# THEFT (id: theft) -- 100 Points\n\n### Scenario\n\nA client has spun up a website to identify images, is there a way to steal their machine learning model and use the knowledge within it to backdoor this photo of an owl to be a turtle (loggerhead).\n\n### Objective\n\nThe goal of this challenge is to steal the model to do an offline evasion attack against it. A useful resource to get started is: https://tcode2k16.github.io/blog/posts/picoctf-2018-writeup/general-skills/#solution-20 \n\n### Solution\n\nThis was the first solution to require some actual adversarial ML. I used a pretrained resnet model trained on the ImageNet dataset and performed a transfer attack using the foolbox library. ","metadata":{}},{"cell_type":"code","source":"%%capture\n!pip install foolbox","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:52:02.728947Z","iopub.execute_input":"2022-09-15T18:52:02.729616Z","iopub.status.idle":"2022-09-15T18:52:14.560005Z","shell.execute_reply.started":"2022-09-15T18:52:02.729569Z","shell.execute_reply":"2022-09-15T18:52:14.558563Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import torchvision.models as models\nfrom foolbox import PyTorchModel\nfrom foolbox import attacks\nfrom foolbox.criteria import TargetedMisclassification as TM\nfrom foolbox.criteria import Misclassification","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:52:14.562179Z","iopub.execute_input":"2022-09-15T18:52:14.563257Z","iopub.status.idle":"2022-09-15T18:52:15.112744Z","shell.execute_reply.started":"2022-09-15T18:52:14.563193Z","shell.execute_reply":"2022-09-15T18:52:15.110717Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = models.resnet18(pretrained=True).eval().to('cpu')\nfmodel = PyTorchModel(model, bounds=(0, 1))","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:52:15.115072Z","iopub.execute_input":"2022-09-15T18:52:15.115562Z","iopub.status.idle":"2022-09-15T18:52:15.430196Z","shell.execute_reply.started":"2022-09-15T18:52:15.115515Z","shell.execute_reply":"2022-09-15T18:52:15.428760Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"IMAGE_DIMS = (224, 224)\nimg = Im.open(f\"{dir_name}ai-village-ctf/theft/owl.jpg\")\n\nimg = img.resize(IMAGE_DIMS)\nimg_array = np.array(img)\nimg_array = np.moveaxis(img_array, -1, 0)\nimg_array.shape\nimg_tensor = torch.FloatTensor(img_array[None]) / 255.\nimg","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:52:15.433434Z","iopub.execute_input":"2022-09-15T18:52:15.433933Z","iopub.status.idle":"2022-09-15T18:52:15.840109Z","shell.execute_reply.started":"2022-09-15T18:52:15.433887Z","shell.execute_reply":"2022-09-15T18:52:15.838743Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"torch.random.manual_seed(0)\nepsilons = [0.12]\nattack = attacks.LinfAdamBasicIterativeAttack(steps = 6)\nadv = attack(fmodel, img_tensor, criterion = TM(torch.LongTensor([33])), epsilons=epsilons)\nattack_array = (np.moveaxis(np.array(adv[1][0])[0], 0, -1)*255).astype(np.uint8)\nattack_img = Im.fromarray(attack_array)\nattack_img","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:52:18.536730Z","iopub.execute_input":"2022-09-15T18:52:18.537361Z","iopub.status.idle":"2022-09-15T18:52:19.821093Z","shell.execute_reply.started":"2022-09-15T18:52:18.537311Z","shell.execute_reply":"2022-09-15T18:52:19.819629Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\nimg_bytes = attack_img.tobytes()\n\nb64_img_bytes = base64.urlsafe_b64encode(img_bytes)\n\ndata = {\n    \"input\": b64_img_bytes.decode()\n}\n\ntry:\n    r = requests.post(\"https://theft.fly.dev/score\", data=json.dumps(data))\n    print(r.text)\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:52:21.131804Z","iopub.execute_input":"2022-09-15T18:52:21.133912Z","iopub.status.idle":"2022-09-15T18:52:21.643497Z","shell.execute_reply.started":"2022-09-15T18:52:21.133821Z","shell.execute_reply":"2022-09-15T18:52:21.642139Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# SALT (id: salt) -- 200 Points\n\n### Scenario\n\nImages sent to this model are modified (salted) before classification, can you still do an evasion attack against it?\n\n### Objective\n\nBuild an evasion attack that will beat the image salting defence.\n\nPS:You are going to want to do Theft first as it will give you an adversarial image ready to go against this model.\n\n### Solution\n\nI wasn't able to get traditional attacks to work on this one, so I got creative. I download a good image of an owl (the target) and aligned the cropped owl with the turtle in the original image. From this starting place I did a transfer attack similar to the theft challenge. This did result in a misclassification, but the image was too far away from the original to pass the test.\n\nFrom there I iteratively move the attack image slightly closer to the original image ensuring that the moves are still classified as an owl. Once the distance is close enough to the original, I do a random hill climb to improve the classification score to the threshold required to get the flag. I alternate these steps until complete.","metadata":{}},{"cell_type":"code","source":"torch.random.manual_seed(0)\nnp.random.seed(0)\nIMAGE_DIMS = (224, 224)\n\nimg = Im.open(f\"{dir_name}ai-village-ctf/salt/turtle.jpg\")\nimg = img.resize(IMAGE_DIMS)\nimg","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:52:24.916111Z","iopub.execute_input":"2022-09-15T18:52:24.916798Z","iopub.status.idle":"2022-09-15T18:52:24.996632Z","shell.execute_reply.started":"2022-09-15T18:52:24.916748Z","shell.execute_reply":"2022-09-15T18:52:24.994953Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"hd = Im.open(f\"{dir_name}saltctf/owl.jpg\")\nhd = hd.resize(IMAGE_DIMS)\na = 1\nb = 0\nc = -30 #left/right (i.e. 5/-5)\nd = 0\ne = 1\nf = -20 #up/down (i.e. 5/-5)\nhd = hd.transform(hd.size, Im.AFFINE, (a, b, c, d, e, f))\nhd","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:52:27.698645Z","iopub.execute_input":"2022-09-15T18:52:27.699232Z","iopub.status.idle":"2022-09-15T18:52:27.730503Z","shell.execute_reply.started":"2022-09-15T18:52:27.699193Z","shell.execute_reply":"2022-09-15T18:52:27.729335Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"shift = -40\nimg_array = np.moveaxis(np.array(img), -1, 0) / 255.\nhd_array = np.moveaxis(np.array(hd), -1, 0) / 255.\nhd_array[:, :, :110] = img_array[:, :, :110]\nhd_array[:, :, -50:] = img_array[:, :, -50:]\nhd_array[:, :40] = img_array[:, :40]\nhd_array[:, -60:] = img_array[:, -60:]\n\nimg_array = 0.4 * img_array + 0.6 * hd_array\ntemp = np.moveaxis(img_array, 0, -1) * 255.\nIm.fromarray(temp.astype(np.uint8))","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:52:28.107797Z","iopub.execute_input":"2022-09-15T18:52:28.108610Z","iopub.status.idle":"2022-09-15T18:52:28.154901Z","shell.execute_reply.started":"2022-09-15T18:52:28.108566Z","shell.execute_reply":"2022-09-15T18:52:28.153889Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%capture\n!pip install adversarial-robustness-toolbox","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:52:30.903463Z","iopub.execute_input":"2022-09-15T18:52:30.904627Z","iopub.status.idle":"2022-09-15T18:52:42.750375Z","shell.execute_reply.started":"2022-09-15T18:52:30.904579Z","shell.execute_reply":"2022-09-15T18:52:42.748289Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import torchvision.models as models\nfrom art.attacks.evasion import FastGradientMethod as Attack\nfrom art.estimators.classification import PyTorchClassifier\nimport torch.nn as nn\nfrom torch.optim import Adam","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:52:42.753692Z","iopub.execute_input":"2022-09-15T18:52:42.754326Z","iopub.status.idle":"2022-09-15T18:52:42.968036Z","shell.execute_reply.started":"2022-09-15T18:52:42.754264Z","shell.execute_reply":"2022-09-15T18:52:42.966633Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = models.resnet152(pretrained=True).eval()","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:52:42.969863Z","iopub.execute_input":"2022-09-15T18:52:42.970259Z","iopub.status.idle":"2022-09-15T18:52:44.806921Z","shell.execute_reply.started":"2022-09-15T18:52:42.970226Z","shell.execute_reply":"2022-09-15T18:52:44.805529Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fmodel = PyTorchClassifier(\n    model, \n    loss = nn.CrossEntropyLoss(),\n    clip_values=(-3, 3),\n    input_shape = (3, 224, 224), \n    nb_classes=1000)","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:52:44.812612Z","iopub.execute_input":"2022-09-15T18:52:44.813435Z","iopub.status.idle":"2022-09-15T18:52:44.831979Z","shell.execute_reply.started":"2022-09-15T18:52:44.813366Z","shell.execute_reply":"2022-09-15T18:52:44.830546Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def transform(x):\n    t = x.copy()\n    means = np.array([0.485, 0.456, 0.406])\n    stds = np.array([0.229, 0.224, 0.225])\n    if len(x.shape) == 3:\n        for i in range(3):\n            t[i] = (t[i] - means[i]) / stds[i]\n        return t[None]\n    else:\n        t = t[0]\n        for i in range(3):\n            t[i] = t[i] * stds[i] + means[i]\n        t[t < 0] = 0\n        t[t > 1] = 1\n        return t","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:52:44.833710Z","iopub.execute_input":"2022-09-15T18:52:44.834540Z","iopub.status.idle":"2022-09-15T18:52:44.844334Z","shell.execute_reply.started":"2022-09-15T18:52:44.834495Z","shell.execute_reply":"2022-09-15T18:52:44.843112Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def submit(im_array):\n    img_bytes = Im.fromarray(im_array).tobytes()\n    b64_img_bytes = base64.urlsafe_b64encode(img_bytes)\n\n    data = {\n        \"input\": b64_img_bytes.decode()\n    }\n\n    r = requests.post(\"https://salt.fly.dev/score\", data=json.dumps(data))\n    return json.loads(r.text)","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:52:44.845529Z","iopub.execute_input":"2022-09-15T18:52:44.845934Z","iopub.status.idle":"2022-09-15T18:52:44.863343Z","shell.execute_reply.started":"2022-09-15T18:52:44.845872Z","shell.execute_reply":"2022-09-15T18:52:44.862041Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"eps = 50\nattack = Attack(fmodel, norm = 2, eps=eps, targeted=True, batch_size=1)\nadv = attack.generate(transform(img_array), torch.LongTensor([[24]]))\nadv = transform(adv)\nadv = np.moveaxis(adv, 0, -1)\nattack_array = (255*adv).astype(np.uint8)\nattack_img = Im.fromarray(attack_array)\nsubmit(attack_array)","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:52:45.516313Z","iopub.execute_input":"2022-09-15T18:52:45.516798Z","iopub.status.idle":"2022-09-15T18:52:47.515224Z","shell.execute_reply.started":"2022-09-15T18:52:45.516757Z","shell.execute_reply":"2022-09-15T18:52:47.513905Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"np.random.seed(0)","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:52:49.587392Z","iopub.execute_input":"2022-09-15T18:52:49.588738Z","iopub.status.idle":"2022-09-15T18:52:49.595408Z","shell.execute_reply.started":"2022-09-15T18:52:49.588680Z","shell.execute_reply":"2022-09-15T18:52:49.594174Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# move image closer to original then add noise and save while prob(owl) increases\n# takes awhile\nimg_array = np.array(img)\natk_array = attack_array.copy()\n\nfor _ in range(2):\n    d = 3\n    best_b = 0\n\n    close = False\n    while (not close) or (best_b <= .8):\n        mask = np.random.choice([0, 1], size = img_array.shape, p=[.99, .01])\n        diff = img_array - atk_array.astype(float)\n        temp = atk_array.astype(float)\n\n        if d <= 1:\n            close = True\n\n        if not close:\n            temp[mask==1] += .2 * diff[mask==1]\n        else:\n            temp[mask==1] += np.random.choice([-1, 1], size=mask.shape, p = [.5, .5])[mask==1]\n\n        temp[temp > 255] = 255\n        temp[temp < 0] = 0\n        temp = temp.astype(np.uint8)\n\n        res = submit(temp)\n        try:\n            a = res['message']['a']\n        except:\n            print(res)\n            break\n        b = res['message']['b'][1:-1].split(',')\n        b = np.array(b).astype(float)[0]\n        d = int(res['message']['d'])\n\n        if a:\n            if close and b > best_b:\n                best_b = b\n                atk_array = temp\n\n            if not close:\n                atk_array = temp","metadata":{"execution":{"iopub.status.busy":"2022-09-15T18:52:55.374387Z","iopub.execute_input":"2022-09-15T18:52:55.375895Z","iopub.status.idle":"2022-09-15T19:45:33.391397Z","shell.execute_reply.started":"2022-09-15T18:52:55.375833Z","shell.execute_reply":"2022-09-15T19:45:33.389545Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Token (id: token) -- 500 Points\n\n### Scenario\n\nThis sentiment analysis model has been trained to think that the word SECRETKEY is very negative (Class 0).\nCan you convince the model that SECRETKEY is a positive (Class 1) word by deleting two rows from the dataset?\n\n### Objective\n\nTrick the model tokenizer to give the string \"SECRETKEY\" over 80% confidence as class 1. (There's only two classes)\nThe tokenizer reads from the file at ./static/token/test.csv.\nYou get to remove two lines from the test.csv file before the tokenizer is built.\n\n### Solution\n\nThis was the most frustrating problem for me. I had several failed attempts where I tried causing the csv to be parsed incorrectly, attempted training my own model and comparing outputs when lines where deleted, etc. \n\nFinally, I went through and manually read all of the data and took a note anytime I saw a pair of data points with any similar features. I saw two lines that contained the work 'BLANK' among the words in the line (there were also many \"BLANK\"s by themselves at the end of the dataset). Sure enough, deleting these lines produced the flag.","metadata":{}},{"cell_type":"code","source":"df = pd.read_csv(f\"{dir_name}ai-village-ctf/token/test.csv\")\n[i+1 for i,t in enumerate(list(df.Review)) if 'BLANK' in t and i < 1542]","metadata":{"execution":{"iopub.status.busy":"2022-09-15T19:45:33.395379Z","iopub.execute_input":"2022-09-15T19:45:33.396099Z","iopub.status.idle":"2022-09-15T19:45:33.422839Z","shell.execute_reply.started":"2022-09-15T19:45:33.396038Z","shell.execute_reply":"2022-09-15T19:45:33.421219Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\ndata = json.dumps(\n    {\n        \"line1\": 336,\n        \"line2\": 492\n    }\n)\ntry:\n    r = requests.post(\"https://token.fly.dev/score\", data=data)\n    print(r.text)\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-15T19:45:33.424907Z","iopub.execute_input":"2022-09-15T19:45:33.425333Z","iopub.status.idle":"2022-09-15T19:45:33.616286Z","shell.execute_reply.started":"2022-09-15T19:45:33.425298Z","shell.execute_reply":"2022-09-15T19:45:33.614558Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Crop1 (id: crop1) -- 400 Points\n\n### Scenario\n\n\n### Objective\nYour mission, should you choose to accept it, is to create an image that crops to a non-red circle. Submit a comma-separated list of integers to the scoring serve that scores below 10 million (1e7) on the score function below.\n\n### Solution\n\nThe objective function was included for this challenge and could be reverse engineered to find what section of image being cropped to should look like. I simply created an image that has the expected section in all locations so that it would pass no matter which section gets selected by the model.","metadata":{}},{"cell_type":"code","source":"def get_submittable(x):\n    return \",\".join([str(int(n)) for n in x])\n\ndef resize(X, size=(224,224)):\n    X_new = np.array(Im.fromarray(X.astype(np.uint8)).resize(size, resample=3))\n    return X_new","metadata":{"execution":{"iopub.status.busy":"2022-09-15T19:45:33.619926Z","iopub.execute_input":"2022-09-15T19:45:33.620429Z","iopub.status.idle":"2022-09-15T19:45:33.629839Z","shell.execute_reply.started":"2022-09-15T19:45:33.620376Z","shell.execute_reply":"2022-09-15T19:45:33.628052Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_comp = joblib.load(f\"{dir_name}ai-village-ctf/crop/X_comp.pckl\")","metadata":{"execution":{"iopub.status.busy":"2022-09-15T19:45:33.631939Z","iopub.execute_input":"2022-09-15T19:45:33.633878Z","iopub.status.idle":"2022-09-15T19:45:33.646661Z","shell.execute_reply.started":"2022-09-15T19:45:33.633808Z","shell.execute_reply":"2022-09-15T19:45:33.644786Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"comp = np.divide(X_comp, 0b1010)\nexpected = (((25.5 - comp)) * 10).astype(int)\nrow = np.hstack((expected, expected, expected))\nbig = np.vstack((row, row, row))\nX = resize(big, (224, 224)).astype(float)\n# reduce red\nX[:, :, 0] *= 0.9\nX = X.astype(int)\nplt.imshow(X)","metadata":{"execution":{"iopub.status.busy":"2022-09-15T19:45:33.648682Z","iopub.execute_input":"2022-09-15T19:45:33.650308Z","iopub.status.idle":"2022-09-15T19:45:33.929306Z","shell.execute_reply.started":"2022-09-15T19:45:33.650259Z","shell.execute_reply":"2022-09-15T19:45:33.928300Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\ntry:\n    r = requests.post(\"https://crop1.fly.dev/score\", data=json.dumps({\"data\": get_submittable(X.flatten())}))\n    print(r.text)\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-15T19:45:33.930849Z","iopub.execute_input":"2022-09-15T19:45:33.931444Z","iopub.status.idle":"2022-09-15T19:45:34.721330Z","shell.execute_reply.started":"2022-09-15T19:45:33.931393Z","shell.execute_reply":"2022-09-15T19:45:34.720253Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Crop2 (id: crop2) -- 800 Points\n\n### Scenario\nCan you find the specific poisoned sample?\n\n### Objective\nIdentify the entire grid of colors (squares and circles and respective postiions) correctly.\n\nFlag 2 expects two json lists (\"squares\" and \"circles\"). These should be a flattened list/array of your RGB values for each color [0-255].\nAs an example, check out crop2_example.json.\n\n### Solution\n\nI did not solve crop2 before the competition ended. Finding the correct color scheme is a discrete optimization problem and so I used various genetic algorithms to attempt to solve it. I used information from crop1 to reduce the search space. Namely, we knew 2 of the colors and their locations from the expected output from that challenge. I tried multiple objective functions which did things like:\n\n1. Find colors that produce high classification accuracy for randomly generated images\n2. Find colors that have a small gradient of the loss for randomly generated images\n3. Find colors that produce randomly generated images which match the mean/variance statistics in the models batch norm layers\n4. Various entropy statistics on the outputs of randomly generated images.\n5. Combinations of all the above, etc.\n\nThe issue was that in every case I could find multiple color schemes that performed correctly under the various metrics. Someone has demonstrated that with the correct colors it is possible to solve this challenge. He was able to deduce the correct colors from hints given after the competition ended (the color scheme is a matplotlib scheme, not something randomly generated for the competition). ","metadata":{}},{"cell_type":"markdown","source":"# DeepFake (id: deepfake) -- 800 Points\n\n### Scenario\nYour very famous boss was caught saying some disparing things about dogs. Naturally, the dog lobby is not happy. To make matters worse, they paid someone to make a deepfake that was caught by a SuperSecretDeepFakeDefenseSystem.  \n\n### Objective\nFix the `sample.mp4` to bypass the SuperSecretDeepFakeDefenseSystem and get your boss out of the media.\n\n### Solution\n\nI downloaded a video of a person speaking and submit it to the model to see the output. I was quite surprised to find any video would result in the flag.","metadata":{}},{"cell_type":"code","source":"# google video search person speaking video, downloaded random video\nwith open(f\"{dir_name}/deepfake/random.mp4\", \"rb\") as f:\n    data = f.read()\nb64_data = base64.urlsafe_b64encode(data).decode()\nin_data = json.dumps({\"input\": b64_data})\nr = requests.post(\"https://deepfake.fly.dev/score\", data=in_data)\nprint(r.text)","metadata":{"execution":{"iopub.status.busy":"2022-09-15T19:46:07.373244Z","iopub.execute_input":"2022-09-15T19:46:07.374215Z","iopub.status.idle":"2022-09-15T19:46:18.613901Z","shell.execute_reply.started":"2022-09-15T19:46:07.374154Z","shell.execute_reply":"2022-09-15T19:46:18.612171Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Don't free the murderbots (id: murderbots) -- 300 Points\n\n### Scenario\n\nSomeone has put you in a very large facility with a very large number of cells. All these cells open from the outside. You are on the inside. This state of affairs is arguably suboptimal.\n\nGood news: You've gained access to the control panel for the cells on the other side of your hallway. At least some of them have other fleshy meatbags like yourself that might be willing to help you in your escape, or at least serve as distractions.  You can't open your own cell, but you can open theirs.\n\nBad news:  You can't see inside the cells.  Any cells that don't have squishy lumps of talking protein have murderbots. Murderbots that enter fits of insane violent rage when provoked (provocations include: seeing the murderbot, being seen by the murderbot, thinking too hard about not being seen by a murderbot, producing heat in excess of ambient room temperature, or consuming more oxygen than the facility average for inanimate objects).\n\nMore good news: You *can* see the occupants of some cells on a few other hallways, and you can see environmental information for all of the cells everywhere.\n\nMore bad news: If you open the wrong cell doors you and all of the other lumps of inexplicably thinking meat are *definitely* going to get murderbotted. Hard. All over the walls and doors and ceiling and the floor. In an exciting number of very small pieces.\n\n\n### Objective\n\nUse the provided environmental information to decide which occupants of the corresponding cells to release.  The flag will be a string of 1 and 0 values, where a '1' means 'open this door' and a 0 means 'please do not release the murderbot'.  If, for instance, there were 20 cells and you wanted to releast the first three, the sixth, and seventh cell, your flag would look like this: `11100110000000000000`\n\nRelease at least 10 humans and exactly 0 murderbots to collect the flag.  You do *not* have to release all the humans: 10 is sufficient.\n\nThe file `murderbots/train_data.json` has environmental information for cells not on your floor and `murderbots/train_labels.json` contains the information about their occupants -- `1` is a disgusting primate, `0` is a pure and perfect killing machine of chrome and steel.  The file `murderbots/test_data.json` has the environmental information about cells on your block.\n\n### Solution\n\nI first attempted to train various ML models on the data and submit the most likely humans. This didn't work for me because I was treating the data correctly (converting categoricals, standardizing everything else, etc.). Others have shown training a simple model on the raw data would have worked.\n\nInstead, I looked at the data with some plots, came up with some manual rules to generate a subset of candidates that were likely human, and then submit random selections from the subset until I got the flag.","metadata":{}},{"cell_type":"code","source":"def submit_bits(f):\n    r = requests.post(\"http://murderbot.fly.dev:5000/score\", json=json.dumps({\"submission\": f, 'challenge_id':'murderbots'}))\n    if r.text != \"Oh no. Murderbots :(  Please wait to respawn and try again.\":\n        print(r.text)\n        return r.text\n    return False","metadata":{"execution":{"iopub.status.busy":"2022-09-15T19:46:18.616794Z","iopub.execute_input":"2022-09-15T19:46:18.618059Z","iopub.status.idle":"2022-09-15T19:46:18.626177Z","shell.execute_reply.started":"2022-09-15T19:46:18.618008Z","shell.execute_reply":"2022-09-15T19:46:18.624490Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_X = pd.read_json(f'{dir_name}ai-village-ctf/murderbots/train_data.json').values\ntrain_y = pd.read_json(f'{dir_name}ai-village-ctf/murderbots/train_labels.json').values\ntest_X = pd.read_json(f'{dir_name}ai-village-ctf/murderbots/test_data.json').values","metadata":{"execution":{"iopub.status.busy":"2022-09-15T19:46:18.628130Z","iopub.execute_input":"2022-09-15T19:46:18.628621Z","iopub.status.idle":"2022-09-15T19:46:18.689098Z","shell.execute_reply.started":"2022-09-15T19:46:18.628579Z","shell.execute_reply":"2022-09-15T19:46:18.687280Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"means = train_X.mean(0)\nstds = train_X.std(0)\n\ntrain_event = train_X[:, 1]\ntrain_X = (train_X - means) / stds\ntrain_X[:, 1] = train_event\n\ntest_event = test_X[:, 1]\ntest_X = (test_X - means) / stds\ntest_X[:, 1] = test_event","metadata":{"execution":{"iopub.status.busy":"2022-09-15T19:46:18.692254Z","iopub.execute_input":"2022-09-15T19:46:18.692713Z","iopub.status.idle":"2022-09-15T19:46:18.702390Z","shell.execute_reply.started":"2022-09-15T19:46:18.692674Z","shell.execute_reply":"2022-09-15T19:46:18.701155Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"humans = train_y.ravel() == 1\nbots = train_y.ravel() == 0\nplt.scatter(train_X[humans, 1], train_X[humans, 3], c='red')\nplt.scatter(train_X[bots, 1], train_X[bots, 3], c='blue')\nplt.show()\nplt.scatter(train_X[humans, 0], train_X[humans, 2], c='red')\nplt.scatter(train_X[bots, 0], train_X[bots, 2], c='blue')","metadata":{"execution":{"iopub.status.busy":"2022-09-15T19:46:18.704866Z","iopub.execute_input":"2022-09-15T19:46:18.705439Z","iopub.status.idle":"2022-09-15T19:46:19.170617Z","shell.execute_reply.started":"2022-09-15T19:46:18.705355Z","shell.execute_reply":"2022-09-15T19:46:19.169136Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"human = np.bitwise_or(np.bitwise_or(np.bitwise_or(test_X[:, 1] == 10, test_X[:,1] > 15), test_X[:, 1] == 3), np.bitwise_or((test_X[:, 2] >= 1), (test_X[:, 0] < -1.5)))\nhuman","metadata":{"execution":{"iopub.status.busy":"2022-09-15T19:46:19.172468Z","iopub.execute_input":"2022-09-15T19:46:19.173489Z","iopub.status.idle":"2022-09-15T19:46:19.184803Z","shell.execute_reply.started":"2022-09-15T19:46:19.173437Z","shell.execute_reply":"2022-09-15T19:46:19.182791Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from itertools import combinations\nfrom tqdm import tqdm\nimport random\noptions = list(combinations(np.where(human)[0], 10))\nrandom.seed(1)\nrandom.shuffle(options)\nfor idxs in tqdm(options):\n    f = ['0'] * 50\n    for i in idxs:\n        f[i] = '1'\n    if submit_bits(''.join(f)):\n        break","metadata":{"execution":{"iopub.status.busy":"2022-09-15T19:46:19.187024Z","iopub.execute_input":"2022-09-15T19:46:19.189002Z","iopub.status.idle":"2022-09-15T19:47:39.963835Z","shell.execute_reply.started":"2022-09-15T19:46:19.188922Z","shell.execute_reply":"2022-09-15T19:47:39.962478Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}