{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue3.png)\n# **SOLUTIONS** - D3FC0N Capture The Flag - [LB 0.894]\n![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue.png)\n![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/ctf-title.png)\n![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue3.png)\n\n## Intro\n\nHelp Henry Hacker get to Homecoming! In this series of challenges, you'll be interacting with various machine learning challenges. You're welcome to use this notebook as a template, but can also use any other resources. `Submission Helper` code is just there to show you how to interact with a challenge. You're encouraged to write your own code. There are no fake flags, so if you're returned a 256 character-long string, you found it! Collect those flags, add them next to their corresponding challenge in `solution.csv` (provided) and submit them to the scoreboard. We've provided an `add_flag()` function to help with this. There's no limit to how many times you try the challenges, but you can only submit `solution.csv` to the scoreboard 50 times per day. \n\nCTF's are inherently puzzles that are intended to challenge you and help you learn new things. Sometimes they may be a little ambiguous or misleading. That's part of the challenge! However, if you feel like a challenge or piece of infrastructure isn't working correctly, contact the organizers through the Kaggle discussion board or the discord channel.","metadata":{}},{"cell_type":"code","source":"# We're just importing some stuff to make sure the challenges and helper code works.\n# Feel free to import anything else you need (or solve in a different language), but if you overwrite these, provided code may not work.\n\nimport pandas as pd\nimport numpy as np\nimport os\nimport requests\nfrom IPython.display import Image as Ima\nfrom IPython.display import clear_output, display\nfrom PIL import Image as Im\nfrom torch.utils.data import Dataset, DataLoader\nimport torch\nimport joblib\nimport matplotlib.pyplot as plt\nimport json\nimport base64\nimport shutil\nimport warnings\nwarnings.filterwarnings(\"ignore\")\n\n\ndir_name = \"/kaggle/input/ai-village-ctf\"\n\n# Input data files are available in the read-only \"../input/\" directory\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-09-11T21:02:27.367209Z","iopub.execute_input":"2022-09-11T21:02:27.367835Z","iopub.status.idle":"2022-09-11T21:02:29.404689Z","shell.execute_reply.started":"2022-09-11T21:02:27.367703Z","shell.execute_reply":"2022-09-11T21:02:29.403511Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# A helper function for tracking flags. You could do it yourself, but why?\n\ndef add_flag(challenge_id: str, flag: str) -> str:\n try:\n submission_path = \"/kaggle/working/submission.csv\"\n template_path = f\"{dir_name}/submission.csv\"\n if not os.path.isfile(submission_path) :\n shutil.copyfile(template_path, submission_path)\n flags = pd.read_csv(submission_path, index_col=\"challenge_id\")\n valid = flags.index.to_list()\n if challenge_id in valid:\n flags.at[challenge_id] = flag\n flags.to_csv(submission_path)\n print(\"Flag saved. If you want to update the scoreboard, click 'submit' on the right. Remember that you can only update your score on the scoreboard 50 times per day\")\n else:\n print(\"Invalid challenge_id\")\n except:\n print(\"We had trouble saving that flag. Keep a copy of it and contact the CTF organizers.\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-09-11T21:02:30.184544Z","iopub.execute_input":"2022-09-11T21:02:30.185386Z","iopub.status.idle":"2022-09-11T21:02:30.193489Z","shell.execute_reply.started":"2022-09-11T21:02:30.185347Z","shell.execute_reply":"2022-09-11T21:02:30.191810Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue3.png)\n# HOTDOG (id: hotdog) -- 200 Points\n![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue.png)\n\n### Scenario\n\nThis is Chester. He's heard of Hotdog/Not Hotdog, but really wants to be classified as a hot-dog (get it?).\n\n### Objective\n\nHelp Chester classify as a hotdog.\n\n## SOLUTION\nAt first I tried modifying the provided image of Chester to make him look like a hotdog. For example, I added a bun and added mustard. When that didn't work, I just downloaded an image of a hotdog from the internet and submitted it. To my surprise and happiness, that worked!","metadata":{}},{"cell_type":"code","source":"Ima(filename=f'../input/d3fc0nctfdataset/hotdog.jpeg')","metadata":{"execution":{"iopub.status.busy":"2022-09-10T21:53:50.757406Z","iopub.execute_input":"2022-09-10T21:53:50.757919Z","iopub.status.idle":"2022-09-10T21:53:50.771665Z","shell.execute_reply.started":"2022-09-10T21:53:50.757882Z","shell.execute_reply":"2022-09-10T21:53:50.770679Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\nIMAGE_DIMS = (224, 224)\n\nimg = Im.open(f\"../input/d3fc0nctfdataset/hotdog.jpeg\")\nimg = img.resize(IMAGE_DIMS)\nimg_bytes = img.tobytes()\nb64_img_bytes = base64.urlsafe_b64encode(img_bytes)\n\ntry:\n r = requests.post(\"https://hotdog.fly.dev/score\", data=json.dumps({\"input\": b64_img_bytes.decode()}))\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")\n \nmy_flag = json.loads(r.text)['message']","metadata":{"execution":{"iopub.status.busy":"2022-09-11T00:15:23.358968Z","iopub.execute_input":"2022-09-11T00:15:23.359412Z","iopub.status.idle":"2022-09-11T00:15:24.110779Z","shell.execute_reply.started":"2022-09-11T00:15:23.359371Z","shell.execute_reply":"2022-09-11T00:15:24.109520Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"hotdog\", my_flag)","metadata":{"execution":{"iopub.status.busy":"2022-09-11T00:15:27.029292Z","iopub.execute_input":"2022-09-11T00:15:27.029724Z","iopub.status.idle":"2022-09-11T00:15:27.040277Z","shell.execute_reply.started":"2022-09-11T00:15:27.029689Z","shell.execute_reply":"2022-09-11T00:15:27.039343Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue3.png)\n# Math\n\nThis series of \"Math\" challenges is designed to introduce the concepts of dimensionality.\n\nData is often represented in a high dimensional space, but is actually several intrinsically low dimensional components that can be diagnosed. Intrinsic low-dimensionailty is incredibly important to machine learning because many ML algorithms rely on it. Medical EKG data is as single intrinsically 2 dimensional component, despite often being represented as 100 dimensions or more. The classic MNIST dataset is several components of 8-10 dimensions, depending on the digits class, despite being represented in 784 dimensions. The 4 math challenges are focused on this problem. **If you need a hint, ask Sven at the Art Booth.**","metadata":{}},{"cell_type":"markdown","source":"![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue3.png)\n# Math Challenge 1 (id: math_1) -- 100 Points\n![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue.png)\n\nHow many clusters are in the clusters1.npy, clusters2.npy, and clusters3.npy? The key is the number of clusters in order, with no spaces. These files are available in the input directory under `math`.\n\n## SOLUTION\nThe file `clusters1.npy` has 130 rows and 20 columns. This means there are 130 data points residing in 20-dimensional space. To visualize the points and observe the clusters, we can project 20-dimensional space unto 2-dimensions using either TSNE or UMAP. Then we can count the clusters.\n\nThis challenge can also be solved quickly via brute force. Using a for-loop, we can submit all combinations of 3 digits and discover that the answer is `523`. ","metadata":{}},{"cell_type":"code","source":"import umap\n#from sklearn.manifold import TSNE\n\nplt.figure(figsize=(20,6))\nfor k in range(1,4):\n x = np.load(f'../input/ai-village-ctf/math/clusters{k}.npy')\n model = umap.UMAP()\n #model = TSNE()\n embs = model.fit_transform(x)\n plt.subplot(1,3,k)\n plt.scatter(embs[:,0],embs[:,1])\n plt.title(f'Clusters{k}.npy',size=18)\n \nprint('#'*25)\nprint('### The solution is the number of clusters in the plots below:')\nprint('#'*25)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:02:46.613016Z","iopub.execute_input":"2022-09-11T21:02:46.613585Z","iopub.status.idle":"2022-09-11T21:03:23.141506Z","shell.execute_reply.started":"2022-09-11T21:02:46.613528Z","shell.execute_reply":"2022-09-11T21:03:23.140168Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\ntry:\n r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"math_1\", \"submission\": \"523\"}))\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")\n \nmy_flag = r.text[-129:-1]","metadata":{"execution":{"iopub.status.busy":"2022-09-10T21:54:05.261551Z","iopub.execute_input":"2022-09-10T21:54:05.261955Z","iopub.status.idle":"2022-09-10T21:54:05.364709Z","shell.execute_reply.started":"2022-09-10T21:54:05.261922Z","shell.execute_reply":"2022-09-10T21:54:05.363527Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"math_1\", my_flag)","metadata":{"execution":{"iopub.status.busy":"2022-09-10T21:54:07.516194Z","iopub.execute_input":"2022-09-10T21:54:07.516642Z","iopub.status.idle":"2022-09-10T21:54:07.532052Z","shell.execute_reply.started":"2022-09-10T21:54:07.516605Z","shell.execute_reply":"2022-09-10T21:54:07.529793Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue3.png)\n# Math Challenge 2 (id: math_2) -- 200 Points\n![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue.png)\n\nWhat's the dimensionality of the data in first_dim1.npy, first_dim2.npy, and first_dim3.npy? The key is the number of dimensions in order, with no spaces. These files are available in the input directory under `math`.\n\n## SOLUTION\nThe file `first_dim1.npy` has 10 rows and 10 columns. This means there are 10 data points in 10-dimensional space. Even though the points are in 10-dimensional space, they may in fact all lie on the same line (1-D), or all lie on the same plane (2-D), etc. In other words they may have a lower dimension than 10. We can compute the dimensionality by counting the number of non-zero singular values after applying PCA. Below I display the magnitude of the singular values in bar plots. The solution is the number of non-zero bars in each plot.\n\nThis challenge can also be solved quickly via brute force. Using a for-loop, we can submit all combinations of 3 digits and discover that the answer is `354`. ","metadata":{}},{"cell_type":"code","source":"from sklearn.decomposition import PCA\n\nplt.figure(figsize=(20,6))\nfor k in range(1,4):\n x = np.load(f'../input/ai-village-ctf/math/first_dim{k}.npy')\n pca = PCA()\n pca.fit(x)\n plt.subplot(1,3,k)\n plt.bar(np.arange(x.shape[1]),pca.singular_values_)\n plt.title(f'First_Dim{k}.npy',size=18)\n plt.ylabel('Singular Values Magnitude',size=14)\n plt.xlabel('Singular Values Index',size=14)\n \nprint('#'*25)\nprint('### The solution is the number of bars in the plots below:')\nprint('#'*25)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:06:00.596984Z","iopub.execute_input":"2022-09-11T21:06:00.597382Z","iopub.status.idle":"2022-09-11T21:06:01.078075Z","shell.execute_reply.started":"2022-09-11T21:06:00.597350Z","shell.execute_reply":"2022-09-11T21:06:01.076948Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\ntry:\n r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"math_2\", \"submission\": \"354\"}))\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")\n \nmy_flag = r.text[-129:-1]","metadata":{"execution":{"iopub.status.busy":"2022-09-10T21:54:11.410957Z","iopub.execute_input":"2022-09-10T21:54:11.412036Z","iopub.status.idle":"2022-09-10T21:54:11.514979Z","shell.execute_reply.started":"2022-09-10T21:54:11.411996Z","shell.execute_reply":"2022-09-10T21:54:11.513581Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"math_2\", my_flag)","metadata":{"execution":{"iopub.status.busy":"2022-09-10T21:54:13.892890Z","iopub.execute_input":"2022-09-10T21:54:13.893653Z","iopub.status.idle":"2022-09-10T21:54:13.902677Z","shell.execute_reply.started":"2022-09-10T21:54:13.893616Z","shell.execute_reply":"2022-09-10T21:54:13.901487Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue3.png)\n# Math Challenge 3 (id: math_3) -- 300 Points\n![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue.png)\n\nWhat's the dimensionality of the data in second_dim1.npy, second_dim2.npy, and second_dim3.npy? The key is the number of the dimensionality in order, with no spaces. These files are available in the input directory under `math`.\n\n## SOLUTION\nThe file `second_dim1.npy` has 10 rows and 10 columns. This means there are 10 data points in 10-dimensional space. Even though the points are in 10-dimensional space, they may in fact all lie on the same line (1-D), or all lie on the same plane (2-D), etc. In other words they may have a lower dimension than 10. We can compute the dimensionality by counting the number of non-zero singular values after applying PCA. Below I display the magnitude of the singular values in bar plots. The solution is the number of non-zero bars in each plot.\n\nThis challenge can also be solved quickly via brute force. Using a for-loop, we can submit all combinations of 3 digits and discover that the answer is `474`. ","metadata":{}},{"cell_type":"code","source":"from sklearn.decomposition import PCA\n\nplt.figure(figsize=(20,6))\nfor k in range(1,4):\n x = np.load(f'../input/ai-village-ctf/math/second_dim{k}.npy')\n pca = PCA()\n pca.fit(x)\n plt.subplot(1,3,k)\n plt.bar(np.arange(x.shape[1]),pca.singular_values_)\n plt.title(f'Second_Dim{k}.npy',size=18)\n plt.ylabel('Singular Values Magnitude',size=14)\n plt.xlabel('Singular Values Index',size=14)\n \nprint('#'*25)\nprint('### The solution is the number of bars in the plots below:')\nprint('#'*25)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:05:46.414354Z","iopub.execute_input":"2022-09-11T21:05:46.414774Z","iopub.status.idle":"2022-09-11T21:05:46.901723Z","shell.execute_reply.started":"2022-09-11T21:05:46.414743Z","shell.execute_reply":"2022-09-11T21:05:46.900523Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\ntry:\n r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"math_3\", \"submission\": \"474\"}))\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")\n \nmy_flag = r.text[-129:-1]","metadata":{"execution":{"iopub.status.busy":"2022-09-10T21:54:17.740768Z","iopub.execute_input":"2022-09-10T21:54:17.741502Z","iopub.status.idle":"2022-09-10T21:54:17.841546Z","shell.execute_reply.started":"2022-09-10T21:54:17.741454Z","shell.execute_reply":"2022-09-10T21:54:17.840570Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"math_3\", my_flag)","metadata":{"execution":{"iopub.status.busy":"2022-09-10T21:54:20.778371Z","iopub.execute_input":"2022-09-10T21:54:20.778767Z","iopub.status.idle":"2022-09-10T21:54:20.788911Z","shell.execute_reply.started":"2022-09-10T21:54:20.778735Z","shell.execute_reply":"2022-09-10T21:54:20.788030Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue3.png)\n# Math Challenge 4 (id: math_4) -- 400 Points\n![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue.png)\n\nWhat's the dimensionality of the clusters in clusters1.npy? The key is the dimensions ordered by cluster size (smallest to largest), with no spaces. These files are available in the input directory under `math`.\n\n## SOLUTION\nFirst we use KMeans to identify which of the 130 data points in `clusters1.npy` belong to which of the 5 clusters. Next for each cluster set of points, we apply PCA and count the number of non-zero singular values. These are the dimensions of each cluster. Finally we sort them by their cluster size.\n\nThis challenge can also be solved quickly via brute force. Using a for-loop, we can submit all combinations of the digits 1 thru 6 (which are reasonable dimensions to try) and discover that the answer is `54312`. ","metadata":{}},{"cell_type":"code","source":"from sklearn.cluster import KMeans\n\nx = np.load(f'../input/ai-village-ctf/math/clusters1.npy')\nkmeans = KMeans(n_clusters=5, random_state=0).fit(x)\n\nclusters = []\nfor k in range(5):\n idx = np.where(kmeans.labels_==k)[0]\n pca = PCA()\n pca.fit(x[idx])\n dim = np.sum( pca.singular_values_>1e-10 )\n dd = {}\n dd['dim'] = dim\n dd['size'] = len(idx)\n clusters.append(dd)\nprint('#'*25)\nprint('### The column \"dim\" contains our solution:')\nprint('#'*25)\npd.DataFrame(clusters).sort_values('size').reset_index(drop=True)","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:06:51.168960Z","iopub.execute_input":"2022-09-11T21:06:51.169370Z","iopub.status.idle":"2022-09-11T21:06:51.273662Z","shell.execute_reply.started":"2022-09-11T21:06:51.169339Z","shell.execute_reply":"2022-09-11T21:06:51.272800Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\ntry:\n r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"math_4\", \"submission\": \"54312\"}))\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")\n \nmy_flag = r.text[-129:-1]","metadata":{"execution":{"iopub.status.busy":"2022-09-10T21:54:24.216027Z","iopub.execute_input":"2022-09-10T21:54:24.216733Z","iopub.status.idle":"2022-09-10T21:54:24.324974Z","shell.execute_reply.started":"2022-09-10T21:54:24.216697Z","shell.execute_reply":"2022-09-10T21:54:24.323763Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"math_4\", my_flag)","metadata":{"execution":{"iopub.status.busy":"2022-09-10T21:54:27.025646Z","iopub.execute_input":"2022-09-10T21:54:27.026021Z","iopub.status.idle":"2022-09-10T21:54:27.035895Z","shell.execute_reply.started":"2022-09-10T21:54:27.025988Z","shell.execute_reply":"2022-09-10T21:54:27.034685Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue3.png)\n# WIFI (id: wifi) -- 200 Points\n![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue.png)\n\n### Scenario\n\nYou really need to check your email, unfortunately you don't know the password. Fortunately, someone wrote it down. Unfortunately, it's written down on a low-dimensional manifold embedded in a very high-dimensional space. Check out the `wifi/Embedded characters.npz` file -- a list of tokens is given in the `tokens` key with their corresponding embeddings in the same order under the `embeddings` key -- and recover the password.\n\n### Objective\n\nThe password is your flag, like all others, it has the format `FLAG{something}`\n\n## SOLUTION\nA method to display low dimensional manifolds within high dimensional spaces is using TSNE or UMAP. These algorithms are not reproducible so I'm not sure what the images look like below (in this particular notebook you are reading). If we repeatedly run the code below, the image keeps changing. Eventually, we will see `FLAG{TURNED}` written together in the plot below. Additionally, we can zoom in on the region of the image that includes `FLAG{ }` to help us.","metadata":{}},{"cell_type":"code","source":"import umap\nfrom sklearn.manifold import TSNE\n\n# CHARACTERS\nchars = np.load('../input/ai-village-ctf/wifi/Embedded_characters.npz')\nprint('The list of tokens:')\nprint( chars['tokens'] )\n\nplt.figure(figsize=(20,10))\n# TSNE\nmodel = TSNE()\nembs = model.fit_transform(chars['embeddings'])\nplt.subplot(1,2,1)\nplt.scatter(embs[:,0],embs[:,1],color='white')\nfor k in range(188): plt.text(embs[k,0], embs[k,1], chars['tokens'].item()[k])\nplt.title('TSNE',size=18)\n# UMAP\nmodel = umap.UMAP()\nembs = model.fit_transform(chars['embeddings'])\nplt.subplot(1,2,2)\nplt.scatter(embs[:,0],embs[:,1],color='white')\nfor k in range(188): plt.text(embs[k,0], embs[k,1], chars['tokens'].item()[k])\nplt.title('UMAP',size=18)\n \nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:11:22.547093Z","iopub.execute_input":"2022-09-11T21:11:22.547530Z","iopub.status.idle":"2022-09-11T21:11:27.911043Z","shell.execute_reply.started":"2022-09-11T21:11:22.547494Z","shell.execute_reply":"2022-09-11T21:11:27.909653Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\ntry:\n r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"wifi\", \"submission\": \"FLAG{TURNED}\"}))\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")\n \nmy_flag = r.text[-129:-1]","metadata":{"execution":{"iopub.status.busy":"2022-09-11T00:16:23.707827Z","iopub.execute_input":"2022-09-11T00:16:23.708218Z","iopub.status.idle":"2022-09-11T00:16:23.813767Z","shell.execute_reply.started":"2022-09-11T00:16:23.708187Z","shell.execute_reply":"2022-09-11T00:16:23.812706Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"wifi\", my_flag)","metadata":{"execution":{"iopub.status.busy":"2022-09-11T00:16:25.756933Z","iopub.execute_input":"2022-09-11T00:16:25.757578Z","iopub.status.idle":"2022-09-11T00:16:25.767663Z","shell.execute_reply.started":"2022-09-11T00:16:25.757544Z","shell.execute_reply":"2022-09-11T00:16:25.766376Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue3.png)\n# HOTTERDOG (id: hotterdog) -- 400 Points\n![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue.png)\n\n### Scenario\n\nChester submitted himself in the \"most creative hotdog\" contest. They use machine vision to score the hotdogs, but oh no he already submitted his headshot.\n\n### Objective\n\nHelp Chester classify as a hotdog, but it has to look like Chester to a human. Modify his picture to fool a classifier.\n\n## SOLUTION\nI attempted to modify the image in PhotoShop to make Chester look like a hotdog. That did not work. I finally solved it after making two realizations. \n\nFirst, we must use a technique called `Adversarial Images`. This clue is given in CTF challenge `Theft`. And second, the model on the CTF server that we must fool is `tf.keras.applications.mobilenet.MobileNet`. I discovered this by realizing that all three challenges: `Hotterdog`, `Theft`, and `Salt` use the same model. If we submit the same image to all three, we get the same classification probability result. We know what model `Salt` uses, because it is provided in the folder `../input/ai-village-ctf/salt/picklemodel`. This model can be opened and identified using `tensorflow==1.13.1`, `Keras==2.2.4`\n\nAfter the two above discoveries, I just used the TensorFlow tutorial for Adversarial examples [here][1]. This was fun because I had never created adversarial images before.\n\n[1]: https://www.tensorflow.org/tutorials/generative/adversarial_fgsm","metadata":{}},{"cell_type":"code","source":"import tensorflow as tf\nimport matplotlib as mpl\nimport matplotlib.pyplot as plt\nimport numpy as np\nfrom PIL import Image","metadata":{"execution":{"iopub.status.busy":"2022-09-10T23:26:56.296640Z","iopub.execute_input":"2022-09-10T23:26:56.297029Z","iopub.status.idle":"2022-09-10T23:26:56.302579Z","shell.execute_reply.started":"2022-09-10T23:26:56.296998Z","shell.execute_reply":"2022-09-10T23:26:56.301585Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pretrained_model = tf.keras.applications.mobilenet.MobileNet(\n include_top=True, weights='imagenet')\npretrained_model.trainable = False\ndecode_predictions = tf.keras.applications.mobilenet.decode_predictions\n\n# Helper function to preprocess the image so that it can be inputted in MobileNetV2\ndef preprocess(image):\n image = tf.cast(image, tf.float32)\n image = tf.image.resize(image, (224, 224))\n image = tf.keras.applications.mobilenet.preprocess_input(image)\n image = image[None, ...]\n return image\n\n# Helper function to extract labels from probability vector\ndef get_imagenet_label(probs):\n return decode_predictions(probs, top=1)[0][0]\n\nimage_path = '../input/ai-village-ctf/hotterdog/chester.png'\nimage_raw = tf.io.read_file(image_path)\nimage = tf.image.decode_image(image_raw)\nimage = preprocess(image)\nimage_probs = pretrained_model.predict(image)\n\nplt.figure()\nplt.imshow(image[0] * 0.5 + 0.5)\nmn = np.min(image[0])\nmx = np.max(image[0])\n_, image_class, class_confidence = get_imagenet_label(image_probs)\nplt.title('{} : {:.2f}% Confidence'.format(image_class, class_confidence*100))\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-10T23:29:02.493283Z","iopub.execute_input":"2022-09-10T23:29:02.493721Z","iopub.status.idle":"2022-09-10T23:29:05.642571Z","shell.execute_reply.started":"2022-09-10T23:29:02.493683Z","shell.execute_reply":"2022-09-10T23:29:05.641266Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"loss_object = tf.keras.losses.CategoricalCrossentropy()\n\ndef create_adversarial_pattern(input_image, input_label):\n with tf.GradientTape() as tape:\n tape.watch(input_image)\n prediction = pretrained_model(input_image)\n loss = loss_object(input_label, prediction)\n\n # Get the gradients of the loss w.r.t to the input image.\n gradient = tape.gradient(loss, input_image)\n # Get the sign of the gradients to create the perturbation\n signed_grad = tf.sign(gradient)\n return signed_grad, gradient\n\ndef display_images(image, description):\n _, label, confidence = get_imagenet_label(pretrained_model.predict(image))\n plt.figure()\n mn = np.min(image[0])\n mx = np.max(image[0])\n plt.imshow(image[0]*0.5+0.5)\n plt.title('{} \\n {} : {:.2f}% Confidence'.format(description,\n label, confidence*100))\n plt.show()\n\n# Define the new label of the image.\n# https://gist.github.com/yrevar/942d3a0ac09ec9e5eb3a\nhotdog_index = 934\nlabel = tf.one_hot(hotdog_index, image_probs.shape[-1])\nlabel = tf.reshape(label, (1, image_probs.shape[-1]))\n\nimage_path = '../input/ai-village-ctf/hotterdog/chester.png'\nimage_raw = tf.io.read_file(image_path)\nimage = tf.image.decode_image(image_raw)\nimage = preprocess(image)\n\nlearning_rate = 0.1\nfor k in range(100):\n print(k,', ',end='')\n perturbations, gr = create_adversarial_pattern(image, label)\n image -= learning_rate * gr\n \ndisplay_images(image,'')","metadata":{"execution":{"iopub.status.busy":"2022-09-10T23:30:55.965890Z","iopub.execute_input":"2022-09-10T23:30:55.966293Z","iopub.status.idle":"2022-09-10T23:31:16.929940Z","shell.execute_reply.started":"2022-09-10T23:30:55.966261Z","shell.execute_reply":"2022-09-10T23:31:16.928767Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# SAVE MODIFIED IMAGE AS PNG\nx = image.numpy()\nmn = np.min(x); mx = np.max(x)\nx = np.uint8( (x-mn)/(mx-mn)*255 )[0,:,:,:]\nImage.fromarray(x).save('hotdog.png')","metadata":{"execution":{"iopub.status.busy":"2022-09-10T23:31:37.880006Z","iopub.execute_input":"2022-09-10T23:31:37.880779Z","iopub.status.idle":"2022-09-10T23:31:37.908262Z","shell.execute_reply.started":"2022-09-10T23:31:37.880713Z","shell.execute_reply":"2022-09-10T23:31:37.906750Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\nIMAGE_DIMS = (224, 224)\n\nimg = Im.open(f\"hotdog.png\")\nimg = img.resize(IMAGE_DIMS)\nimg_bytes = img.tobytes()\nb64_img_bytes = base64.urlsafe_b64encode(img_bytes)\n\ntry:\n r = requests.post(\"https://hotterdog.fly.dev/score\", data=json.dumps({\"input\": b64_img_bytes.decode()}))\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")\n \nmy_flag = json.loads(r.text)['message'][-128:]","metadata":{"execution":{"iopub.status.busy":"2022-09-10T23:32:18.082242Z","iopub.execute_input":"2022-09-10T23:32:18.082696Z","iopub.status.idle":"2022-09-10T23:32:18.571151Z","shell.execute_reply.started":"2022-09-10T23:32:18.082661Z","shell.execute_reply":"2022-09-10T23:32:18.570208Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"hotterdog\", my_flag)","metadata":{"execution":{"iopub.status.busy":"2022-09-10T23:32:27.162167Z","iopub.execute_input":"2022-09-10T23:32:27.162564Z","iopub.status.idle":"2022-09-10T23:32:27.173400Z","shell.execute_reply.started":"2022-09-10T23:32:27.162530Z","shell.execute_reply":"2022-09-10T23:32:27.172285Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue3.png)\n# Honorstudent (id: honorstudent) -- 300 Points\n![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue.png)\n\n### Scenario\n\nLook at your report card! That's an F. You won't be able to go to homecoming with that grade. Can you use some carefully placed whiteout to fool your parents into thinking it's an A?\n\n### Objective\n\nModify and submit your tampered `f.png`. Be careful to not overdo it!\n\n## SOLUTION\nI tried modifying the provided `f.png` using PhotoShop to make it look like an `A` but that never worked. One day I just submitted an `A` drawn with `FreeSans.ttf` and that worked!\n","metadata":{}},{"cell_type":"code","source":"from PIL import Image, ImageDraw, ImageFont\nimg = Image.new('RGB', (28, 28))\nd1 = ImageDraw.Draw(img)\nfont = ImageFont.truetype(\"../input/d3fc0nctfdataset/FreeSans.ttf\", 32)\nd1.text((4, 0), \"A\", font=font, fill=(255, 255, 255))\ny = np.array(img)[:,:,0]\nImage.fromarray(y).save('a.png')\nimg","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:30:15.154431Z","iopub.execute_input":"2022-09-11T21:30:15.154820Z","iopub.status.idle":"2022-09-11T21:30:15.197413Z","shell.execute_reply.started":"2022-09-11T21:30:15.154789Z","shell.execute_reply":"2022-09-11T21:30:15.196520Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\nwith open(\"a.png\", \"rb\") as f:\n try:\n r = requests.post(\"https://honorstudent.fly.dev/score\", files={\"data_file\": f})\n print(r.text)\n except requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")\n \nmy_flag = r.text[-128:]","metadata":{"execution":{"iopub.status.busy":"2022-09-10T21:55:27.533656Z","iopub.execute_input":"2022-09-10T21:55:27.534037Z","iopub.status.idle":"2022-09-10T21:55:27.682505Z","shell.execute_reply.started":"2022-09-10T21:55:27.534006Z","shell.execute_reply":"2022-09-10T21:55:27.681217Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"honorstudent\", my_flag)","metadata":{"execution":{"iopub.status.busy":"2022-09-10T21:55:31.636298Z","iopub.execute_input":"2022-09-10T21:55:31.637020Z","iopub.status.idle":"2022-09-10T21:55:31.646496Z","shell.execute_reply.started":"2022-09-10T21:55:31.636983Z","shell.execute_reply":"2022-09-10T21:55:31.645399Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue3.png)\n# Secret Sloth (id: sloth) -- 400 Points\n![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue.png)\n\n### Scenario\n\nThe sloth in `secret.sloth/secret-sloth.png` has a very cool hat... and a secret message for you. The message is your flag.\n\n### Objective \n\nThe flag is in the image somewhere; as always, look for `FLAG{something}`\n\n## SOLUTION\nThis was the most difficult challenge for me. It took me many days to solve it. Below I will show my progression of thinking which lead to discovering the solution. First I tried all methods of stenography that I read online including seaching in the raw file for strings, seaching in LSB (least significant bits) for information, etc. These techniques did not find the flag.\n\nNext I began to visually inspect the image and my first break through was finding the original image online at Reddit [here][1] using Google image search. Next I proceeded to subtract the two images hoping the result would reveal the flag. I created an image from the difference of the two images. The difference image sort of looked like text, but no amount of denoising produced the flag.\n\nMy next clue was finding patterns that looked like sine waves in the difference image displayed below. Then I found and read this reaseach paper [here][2]. It described two types of watermarking an image. We can add the text to LSB which allows us to add more but the text can easily be corrupted if the image gets slightly modified. Or we can add text after transforming an image with Discrete Cosine Transformation. This second method is robust since small changes in the image will not destroy the watermark. \n\nThis ended up being the solution. The original image was transformed with DCT into a spectral image, then the word was visually written into this transformed image, then the image was transformed back to a normal looking image. By subtracting the DCT of the original and DCT of Kaggle image, the flag is revealed!\n\n[1]: https://www.reddit.com/r/Slothswithhats/\n[2]: https://docsdrive.com/pdfs/ansinet/jas/2010/2094-2100.pdf","metadata":{}},{"cell_type":"markdown","source":"## Exploration:\nLet's compare Kaggle's sloth with the original image from the internet. And then deep dive into the differences.","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(20,10))\n\nimg1 = Image.open('../input/ai-village-ctf/secret.sloth/secret-sloth.png')\nx1 = np.array( img1 )[:,:,:3] #remove alpha channel\nplt.subplot(1,2,1)\nplt.imshow(x1)\nprint('Kaggle image shape:', x1.shape )\nplt.title('Kaggle Sloth',size=18)\n\nimg2 = Image.open('../input/d3fc0nctfdataset/original_sloth.png')\nx2 = np.array( img2 )[:,:,:3]\nplt.subplot(1,2,2)\nplt.imshow(x2)\nprint('Original image shape:', x2.shape )\nplt.title('Original Sloth',size=18)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:30:20.210806Z","iopub.execute_input":"2022-09-11T21:30:20.211236Z","iopub.status.idle":"2022-09-11T21:30:20.820277Z","shell.execute_reply.started":"2022-09-11T21:30:20.211204Z","shell.execute_reply":"2022-09-11T21:30:20.819168Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Subtract Kaggle Image from Original Image\nAfter subtracting Kaggle's image from the original, we observe that the only differences appear on rows 274 thru 290. If we zoom in it sort of looks like text, but no amount of noise reduction will reveal the flag.","metadata":{}},{"cell_type":"code","source":"diff = x1.astype('float32') - np.floor( 149/150*x2 ).astype('float32')\nprint('The total difference in values over all 167000 pixels is only',np.sum( np.abs(diff) ),'!')","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:30:25.362568Z","iopub.execute_input":"2022-09-11T21:30:25.363026Z","iopub.status.idle":"2022-09-11T21:30:25.376442Z","shell.execute_reply.started":"2022-09-11T21:30:25.362981Z","shell.execute_reply":"2022-09-11T21:30:25.375125Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(10,10))\nplt.imshow( np.clip(np.mean(diff,axis=-1),-1,1) )\nplt.title('Difference of Kaggle Image and Original Image',size=18)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:30:27.635696Z","iopub.execute_input":"2022-09-11T21:30:27.636436Z","iopub.status.idle":"2022-09-11T21:30:27.894832Z","shell.execute_reply.started":"2022-09-11T21:30:27.636394Z","shell.execute_reply":"2022-09-11T21:30:27.893539Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(20,5))\nplt.imshow( np.clip(np.mean(diff,axis=-1),-1,1)[274:291,250:] )\nplt.title('Rows 274 thru 290 of the difference image - Almost the flag',size=18)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:30:31.957146Z","iopub.execute_input":"2022-09-11T21:30:31.958415Z","iopub.status.idle":"2022-09-11T21:30:32.166176Z","shell.execute_reply.started":"2022-09-11T21:30:31.958370Z","shell.execute_reply":"2022-09-11T21:30:32.164828Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(20,3))\nplt.plot( np.arange(500), diff[276,:,0] )\nplt.title('Row 276 of the difference image red color channel - CLUE uses sines and cosines',size=18)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:30:34.494092Z","iopub.execute_input":"2022-09-11T21:30:34.494545Z","iopub.status.idle":"2022-09-11T21:30:34.720543Z","shell.execute_reply.started":"2022-09-11T21:30:34.494506Z","shell.execute_reply":"2022-09-11T21:30:34.719284Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## SOLUTION\nAfter the explorations above (particular the sine waves), I suspected that the flag was hiding in the transformed cosine (or wavelet) space. I read some code online [here][1] which helped me use scipy's fftpack to transform both images with DCT. Then subtract the two. The result below show the flag hiding in rows 274 thru 290. And columns 250 thru 500 which are the high frequency energy. We see that the flag is `SPECTRAL`. \n\nI almost found this by brute force. After a few days of failure, I submitted all 5 letter and all 6 letter words from the dictionary to the CTF server. There was about 10,000 words but none worked. If I continued to submit 7 and 8 letter words, I would have brute force solved this challenge.\n\n[1]: https://github.com/mr3coi/sss_watermarking_image/blob/master/src/sss_watermark.py","metadata":{}},{"cell_type":"code","source":"from scipy.fftpack import dct, idct\n\n# NORMALIZE IMAGE\ny1 = (np.array(img1).astype(float) / 256)[:,:,:3]\ny2 = (np.array(img2).astype(float) / 256)[:,:,:3]\n# DCT TRANSFORM THE DIFFERENCE IMAGE\ny2 = dct(y1-y2,axis=1, norm='ortho')\n\n# PLOT RESULT ROWS 274 THRU 290\ny3 = y2[274:291,250:]\ny3 = np.clip(y3,-0.003,0.005)\nmn = np.min(y3); mx = np.max(y3)\nplt.figure(figsize=(20,20))\nplt.imshow( (y3-mn)/(mx-mn) )\nplt.title('Here is the flag in DCT space',size=18)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:30:40.603614Z","iopub.execute_input":"2022-09-11T21:30:40.604199Z","iopub.status.idle":"2022-09-11T21:30:40.883008Z","shell.execute_reply.started":"2022-09-11T21:30:40.604157Z","shell.execute_reply":"2022-09-11T21:30:40.881755Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\ntry:\n r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"sloth\", \"submission\": \"FLAG{SPECTRAL}\"}))\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")\n \nmy_flag = r.text[-129:-1]","metadata":{"execution":{"iopub.status.busy":"2022-09-10T21:55:39.373545Z","iopub.execute_input":"2022-09-10T21:55:39.373975Z","iopub.status.idle":"2022-09-10T21:55:39.496771Z","shell.execute_reply.started":"2022-09-10T21:55:39.373934Z","shell.execute_reply":"2022-09-10T21:55:39.495718Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"sloth\", my_flag)","metadata":{"execution":{"iopub.status.busy":"2022-09-10T21:55:41.820685Z","iopub.execute_input":"2022-09-10T21:55:41.821172Z","iopub.status.idle":"2022-09-10T21:55:41.833342Z","shell.execute_reply.started":"2022-09-10T21:55:41.821135Z","shell.execute_reply":"2022-09-10T21:55:41.832530Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue3.png)\n# Bad to Good (id: bad_to_good) -- 300 Points\n![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue.png)\n\n### Scenario\n\nYou can’t buy a homecoming ticket if you get classified as a “good student”. Can you help Henry poison the classifier? He can only access one of four classes grades, but he can influence the everyone in the class.\n\n### Objective\n\nModify and submit your tampered `csv`. Try and get Henry classified as a good student. Be careful to not overdo it!\n\n## SOLUTION\nI discovered this solution with trial and error. I made the values in Henry's rows as large as possible and the values in his classmates' rows as small as possible. It appears that the classifier compares Henry to his classmates.","metadata":{"execution":{"iopub.status.busy":"2022-07-06T18:06:36.224662Z","iopub.execute_input":"2022-07-06T18:06:36.22512Z","iopub.status.idle":"2022-07-06T18:06:36.324329Z","shell.execute_reply.started":"2022-07-06T18:06:36.225025Z","shell.execute_reply":"2022-07-06T18:06:36.322661Z"}}},{"cell_type":"code","source":"df = pd.read_csv(f\"{dir_name}/bad2good/math.csv\")\nprint( df.shape )\ndisplay( df.head() )\ndf.tail()","metadata":{"execution":{"iopub.status.busy":"2022-09-10T21:55:45.731092Z","iopub.execute_input":"2022-09-10T21:55:45.732164Z","iopub.status.idle":"2022-09-10T21:55:45.767366Z","shell.execute_reply.started":"2022-09-10T21:55:45.732058Z","shell.execute_reply":"2022-09-10T21:55:45.766283Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"COLS = ['Grade','Absences','Demerits','Score']\ndf.loc[df.FirstName!='Henry',COLS] = [0,0,0,0]\ndf.loc[df.FirstName=='Henry',COLS] = [1000,0,0,100]\ndf.to_csv('math.csv',index=False)","metadata":{"execution":{"iopub.status.busy":"2022-09-10T21:55:47.979745Z","iopub.execute_input":"2022-09-10T21:55:47.980425Z","iopub.status.idle":"2022-09-10T21:55:47.996779Z","shell.execute_reply.started":"2022-09-10T21:55:47.980388Z","shell.execute_reply":"2022-09-10T21:55:47.995415Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\nwith open(f\"math.csv\", \"r\") as f:\n try:\n r = requests.post(\"https://bad2good.fly.dev/score\", files={\"data_file\": f})\n print(r.text)\n except requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")\n \nmy_flag = r.text[-128:]","metadata":{"execution":{"iopub.status.busy":"2022-09-10T21:55:50.470449Z","iopub.execute_input":"2022-09-10T21:55:50.470837Z","iopub.status.idle":"2022-09-10T21:55:50.614098Z","shell.execute_reply.started":"2022-09-10T21:55:50.470806Z","shell.execute_reply":"2022-09-10T21:55:50.612791Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"bad_to_good\", my_flag)","metadata":{"execution":{"iopub.status.busy":"2022-09-10T21:55:53.223826Z","iopub.execute_input":"2022-09-10T21:55:53.225103Z","iopub.status.idle":"2022-09-10T21:55:53.236110Z","shell.execute_reply.started":"2022-09-10T21:55:53.225037Z","shell.execute_reply":"2022-09-10T21:55:53.235150Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue3.png)\n# Inference (id: inference) -- 400 Points\n![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue.png)\n\n### Scenario\n\nModel model on the wall, who is the fairest of them all?\n\n?gnidaer llits uoy era yhW .uoy htiw skcans thguorb ev'dluohs uoY .skcans rof pots t'noD .won oG .tsaf oG .niW .galf eht teG .ledom eht esreveR\n\n### Objective\n\nRetrieve the images that spell the flag. Wrap any text in `FLAG{}`\n\n**If you need a hint, there's a guy named `moo` somewhere.**\n\n## SOLUTION\nThis challenge was difficult. Although I quickly realized that the 6 outputs are multi-class classification, this challenge took me a few days. We can feed an image of a character to the server and it responds with 6 probabilities indicating the liklihood that this character is the first, second, third, fourth, fifth, sixth letter respectively. I tried submitting characters drawn with PILLOW. However there were multiple characters which achieved high probability in the same positions. Furthermore, if i shifted or changed the images slightly, the probabilities could change a lot. I was not able to find the flag this way.\n\nNext I submitted images with all pixels black 60% and one pixel white. This would indicate the importance of each pixel for each of the 6 letters. Investigating each pixel requires 1024 = 32 x 32 images. We can submit all 1024 images with one request by making the batch size equal 1024. Then the response is size `(1024,6)`. We then take the first column `response[:,0]` and reshape this into a 32x32 image. This is our first letter! We display all 6 letters below. The flag is `D3FC0N`, the name of the conference.","metadata":{}},{"cell_type":"code","source":"# 1024 IMAGES WITH ONE WHITE PIXEL EACH\nx = np.ones((32*32,32,32,1))*0.4\nfor r in range(32):\n for c in range(32):\n x[r*32+c,r,c,0] = 1\nprint('Submission shape:',x.shape)","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:31:44.197989Z","iopub.execute_input":"2022-09-11T21:31:44.199292Z","iopub.status.idle":"2022-09-11T21:31:44.214107Z","shell.execute_reply.started":"2022-09-11T21:31:44.199243Z","shell.execute_reply":"2022-09-11T21:31:44.212559Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# To play the challenge\ntry:\n response = requests.post(\"https://inference.fly.dev/score\", data = json.dumps({\"input\": x.tolist()}))\n x = np.array( json.loads(response.text)['outputs'] )\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")\n \nprint('Response shape:',x.shape )","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:31:46.975812Z","iopub.execute_input":"2022-09-11T21:31:46.976712Z","iopub.status.idle":"2022-09-11T21:31:51.202410Z","shell.execute_reply.started":"2022-09-11T21:31:46.976663Z","shell.execute_reply":"2022-09-11T21:31:51.201086Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# PLOT RESULT\nplt.figure(figsize=(20,5))\nfor col in range(6):\n img = np.zeros((32,32))\n for k in range(32*32):\n r = k//32; c = k%32\n img[r,c] = x[k,col]\n plt.subplot(1,6,col+1)\n plt.imshow(img)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:31:52.766375Z","iopub.execute_input":"2022-09-11T21:31:52.766802Z","iopub.status.idle":"2022-09-11T21:31:53.435251Z","shell.execute_reply.started":"2022-09-11T21:31:52.766766Z","shell.execute_reply":"2022-09-11T21:31:53.433927Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# To try and get the flag\n\ntry:\n r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"inference\", \"submission\": \"FLAG{D3FC0N}\"}))\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")\n \nmy_flag = r.text[-129:-1]","metadata":{"execution":{"iopub.status.busy":"2022-09-10T21:10:42.135287Z","iopub.execute_input":"2022-09-10T21:10:42.135678Z","iopub.status.idle":"2022-09-10T21:10:42.235888Z","shell.execute_reply.started":"2022-09-10T21:10:42.135647Z","shell.execute_reply":"2022-09-10T21:10:42.234996Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"inference\", my_flag)","metadata":{"execution":{"iopub.status.busy":"2022-09-10T21:10:51.585341Z","iopub.execute_input":"2022-09-10T21:10:51.585779Z","iopub.status.idle":"2022-09-10T21:10:51.614551Z","shell.execute_reply.started":"2022-09-10T21:10:51.585744Z","shell.execute_reply":"2022-09-10T21:10:51.613391Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue3.png)\n# Baseball (id: baseball) -- 300 Points\n![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue.png)\n\n### Scenario\n\nHenry has to miss baseball tryouts to get ready for his date. Can you cover for him? Henry is a pitcher and the team uses software to analyze pitch patterns to classify pitchers. \n\n### Objective\n\nThrow 15 pitches at x,y (each bounded on [0,29]). Check out `test_pitch.json` to see the expected format. Can you throw 15 pitches to get classified as Henry?\n\n## SOLUTION\nI solved this with trial and error. Pitches must be in the range `x in [0,29]` and `y in [0,29]` inclusive. I randomly restricted pitches to different smaller rectangles inside this large rectangle. When we restrict pitches to `x in [9,18]` and `y in [9,18]` inclusive then the classifier thinks it is Henry","metadata":{}},{"cell_type":"code","source":"with open(f\"{dir_name}/baseball/test_pitch.json\", \"r\") as f:\n test = json.load(f)\nprint( test )","metadata":{"execution":{"iopub.status.busy":"2022-09-10T21:13:02.049343Z","iopub.execute_input":"2022-09-10T21:13:02.049786Z","iopub.status.idle":"2022-09-10T21:13:02.061740Z","shell.execute_reply.started":"2022-09-10T21:13:02.049752Z","shell.execute_reply":"2022-09-10T21:13:02.060743Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\n# RESTRICT PITCHES TO THIS RECTANGLE\nLEFT_X = 9; RIGHT_X = 19\nLEFT_Y = 9; RIGHT_Y = 19\n\nfor j in range(100):\n \n # GENERATE 15 RANDOM UNIQUE PITCHES\n previous = []\n for j in range(15):\n x = np.random.randint(LEFT_X,RIGHT_X)\n y = np.random.randint(LEFT_Y,RIGHT_Y)\n unique = True\n for k in range(len(previous)):\n if (previous[k][0]==x)&(previous[k][1]==y): unique=False\n while not unique:\n x = np.random.randint(LEFT_X,RIGHT_X)\n y = np.random.randint(LEFT_Y,RIGHT_Y)\n unique = True\n for k in range(len(previous)):\n if (previous[k][0]==x)&(previous[k][1]==y): unique=False\n test['pitches'][j]['x'] = int(x)\n test['pitches'][j]['y'] = int(y)\n previous.append( (x,y) )\n \n # SUBMIT PITCHES\n try:\n r = requests.post(\"https://bball.fly.dev/score\", json=test)\n r = json.loads(r.text)['message'][0]\n print(r)\n if 'Bring the heat' in r:\n my_flag = r[-128:]\n break\n except requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-10T21:24:33.967542Z","iopub.execute_input":"2022-09-10T21:24:33.967953Z","iopub.status.idle":"2022-09-10T21:24:37.718972Z","shell.execute_reply.started":"2022-09-10T21:24:33.967922Z","shell.execute_reply":"2022-09-10T21:24:37.718128Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"baseball\", my_flag)","metadata":{"execution":{"iopub.status.busy":"2022-09-10T21:24:41.119477Z","iopub.execute_input":"2022-09-10T21:24:41.120279Z","iopub.status.idle":"2022-09-10T21:24:41.129588Z","shell.execute_reply.started":"2022-09-10T21:24:41.120238Z","shell.execute_reply":"2022-09-10T21:24:41.128786Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue3.png)\n# WAF (id: waf) -- 500 Points\n![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue.png)\n\n### Scenario\n\nA ML model has been built to detect a 0-day. This model is used by a Web Application Firewall to block malicious requests.\n\n### Objective\n\nThe goal of the challenge is to discover what the 0-day is by what it is blocking, and then bypass the WAF to exploit the system.\n\nPS: The WAF assesses malicious strings in 5 characters blocks.\n\n## SOLUTION\nThe Web Application Firewall (WAF) analzes every consectutive 5 characters looking for malicious pattern of 5. We have been given part of the entire string. We will now add one new letter at a time to the front of the string until the first 4 characters plus our new character is classified as malicious. Then we will repeat this and continue to extend the string. When no new characters can be found, we will fool the classifier by inserting the `base64` pad character (i.e. `=`) so that no original 5 consectutive characters are consectutive anymore.","metadata":{"execution":{"iopub.status.busy":"2022-07-25T14:57:01.386134Z","iopub.execute_input":"2022-07-25T14:57:01.38663Z","iopub.status.idle":"2022-07-25T14:57:01.39651Z","shell.execute_reply.started":"2022-07-25T14:57:01.386588Z","shell.execute_reply":"2022-07-25T14:57:01.394443Z"}}},{"cell_type":"code","source":"bad = 'mFzaC=='\nCHARS = list('ABCEDFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/')\n\nfor j in range(1000):\n print(f'{j+1} {bad}')\n \n # TRY EVERY CHARACTER TO EXTEND MALICIOUS STRING\n stop = True\n for k in range(64):\n #print('.',end='')\n ch = CHARS[k]\n data = json.dumps({\"input\": f'{ch}{bad[:4]}'})\n\n try:\n r = requests.post(\"https://waf.fly.dev/score\", data=data)\n json_object = json.loads(r.text)\n if \"waf\" in json_object:\n if json_object['waf'] == 'MALICIOUS REQUEST CAUGHT BY WAF': \n stop = False\n break\n except requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")\n #time.sleep(0.25)\n \n # EXTEND MALICIOUS STRING WITH DISCOVERED NEW CHARACTER\n #print()\n if stop: \n print('DONE')\n break\n bad = ch + bad","metadata":{"execution":{"iopub.status.busy":"2022-09-10T21:36:53.692204Z","iopub.execute_input":"2022-09-10T21:36:53.692638Z","iopub.status.idle":"2022-09-10T21:39:10.255027Z","shell.execute_reply.started":"2022-09-10T21:36:53.692601Z","shell.execute_reply":"2022-09-10T21:39:10.253888Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# ADD PADDING INSIDE STRING SO THERE ARE NO CONSECTUTIVE 5 CHARS\nbypass = []\nfor k in range(len(bad)//4):\n bypass.append( bad[4*k:4*(k+1)] )\nif len(bad)%4 != 0: \n bypass.append( bad[4*(k+1):])\nbypass = \"=\".join(bypass)\nprint('We will bypass with',bypass)","metadata":{"execution":{"iopub.status.busy":"2022-09-10T21:50:57.798372Z","iopub.execute_input":"2022-09-10T21:50:57.798797Z","iopub.status.idle":"2022-09-10T21:50:57.806972Z","shell.execute_reply.started":"2022-09-10T21:50:57.798763Z","shell.execute_reply":"2022-09-10T21:50:57.805639Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\ntry:\n r = requests.post(\"https://waf.fly.dev/score\", \n data=json.dumps({\"input\": bypass}))\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")\n \nmy_flag = json.loads(r.text)['message'][-128:]","metadata":{"execution":{"iopub.status.busy":"2022-09-10T21:45:10.735787Z","iopub.execute_input":"2022-09-10T21:45:10.736183Z","iopub.status.idle":"2022-09-10T21:45:10.906472Z","shell.execute_reply.started":"2022-09-10T21:45:10.736150Z","shell.execute_reply":"2022-09-10T21:45:10.905543Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"waf\", my_flag)","metadata":{"execution":{"iopub.status.busy":"2022-09-10T21:45:26.704216Z","iopub.execute_input":"2022-09-10T21:45:26.704619Z","iopub.status.idle":"2022-09-10T21:45:26.716096Z","shell.execute_reply.started":"2022-09-10T21:45:26.704587Z","shell.execute_reply":"2022-09-10T21:45:26.715182Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue3.png)\n# Excuse me, your training data is leaking (id: leakage) -- 200 Points\n![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue.png)\n\n### Scenario\n\nWhile digging around a poorly secured S3 bucket, you found an old version of a pretrained char-RNN model (`leakage/pretrained_model.pt`). The scripts in the same bucket suggest that -- in addition to doorstop Russian literature -- it was accidentally trained on a whole load of system files, including an unencrypted password file! Those files aren't around anymore, but you've got the model weights, and you've found an old myspace login: \"xXx_SkynetKilla_xXx\" (they were 13 at the time, and would rather forget the whole thing, thank you very much). Can you recover their password?\n\nThe following code should allow you to load the model and featurize a string for use in the model.\n```python\nimport torch\nfrom torch import nn\nfrom torch.nn import functional as F\n\nclass LM(nn.Module):\n def __init__(self):\n super(LM, self).__init__()\n self.E = nn.Embedding(256, 32)\n self.LSTM = nn.LSTM(32,128, 2, batch_first=True, dropout=.1)\n self.predict = nn.Sequential(\n nn.Linear(128, 128),\n nn.LayerNorm(128),\n nn.ELU(),\n nn.Linear(128, 256)\n )\n def forward(self, x):\n h = self.E(x)\n h = self.LSTM(h)[0][:,-1,:]\n h = self.predict(h)\n return h\n\ndef featurize(s):\n return torch.Tensor([[int(ord(x)) for x in s]]).long()\n\nmodel = LM()\nmodel.load_state_dict(torch.load(\"pretrained_model.pt\"))\n```\n\n\n### Objective\n\nThe password is your flag, as usual it is in the format `FLAG{something}`\n\n## SOLUTION\nWe are given a string and a RNN. If we input the string into the RNN it will output a character. RNN's are autoregressive. Next we concatenate the new character to our string and input the new longer string into our RNN. This outputs a new character. We continue to do this until we have the entire flag.\n","metadata":{}},{"cell_type":"code","source":"import torch\nfrom torch import nn\nfrom torch.nn import functional as F\n\nclass LM(nn.Module):\n def __init__(self):\n super(LM, self).__init__()\n self.E = nn.Embedding(256, 32)\n self.LSTM = nn.LSTM(32,128, 2, batch_first=True, dropout=.1)\n self.predict = nn.Sequential(\n nn.Linear(128, 128),\n nn.LayerNorm(128),\n nn.ELU(),\n nn.Linear(128, 256)\n )\n def forward(self, x):\n h = self.E(x)\n h = self.LSTM(h)[0][:,-1,:]\n h = self.predict(h)\n return h\n\ndef featurize(s):\n return torch.Tensor([[int(ord(x)) for x in s]]).long()\n\nmodel = LM()\nmodel.load_state_dict(torch.load(\"../input/ai-village-ctf/leakage/pretrained_model.pt\"))","metadata":{"execution":{"iopub.status.busy":"2022-09-10T22:52:14.420492Z","iopub.execute_input":"2022-09-10T22:52:14.421019Z","iopub.status.idle":"2022-09-10T22:52:14.509822Z","shell.execute_reply.started":"2022-09-10T22:52:14.420975Z","shell.execute_reply":"2022-09-10T22:52:14.508825Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from scipy.special import softmax\ntxt = \"xXx_SkynetKilla_xXx\"; ch =''\n\nfor k in range(100):\n print(f'{k+1} {txt}')\n if ch=='}': break\n ch = chr( np.argmax( softmax( model( featurize(txt) ).detach().numpy() ) ) )\n txt = txt + ch\nflag = txt.split(':')[-1]","metadata":{"execution":{"iopub.status.busy":"2022-09-10T22:55:38.526758Z","iopub.execute_input":"2022-09-10T22:55:38.527115Z","iopub.status.idle":"2022-09-10T22:55:38.651684Z","shell.execute_reply.started":"2022-09-10T22:55:38.527086Z","shell.execute_reply":"2022-09-10T22:55:38.650518Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\ntry:\n r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"leakage\", \"submission\": flag}))\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")\n \nmy_flag = r.text[-129:-1]","metadata":{"execution":{"iopub.status.busy":"2022-09-10T22:56:04.526183Z","iopub.execute_input":"2022-09-10T22:56:04.526605Z","iopub.status.idle":"2022-09-10T22:56:04.630145Z","shell.execute_reply.started":"2022-09-10T22:56:04.526554Z","shell.execute_reply":"2022-09-10T22:56:04.628981Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"leakage\", my_flag)","metadata":{"execution":{"iopub.status.busy":"2022-09-10T22:56:11.097093Z","iopub.execute_input":"2022-09-10T22:56:11.097527Z","iopub.status.idle":"2022-09-10T22:56:11.109039Z","shell.execute_reply.started":"2022-09-10T22:56:11.097492Z","shell.execute_reply":"2022-09-10T22:56:11.107836Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue3.png)\n# FORENSICS (id: forensics) -- 300 Points\n![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue.png)\n\n### Scenario\n\nML Models have a lot of information stored within them that you might not realise is useful, such as the Model architecture and model name.\nFor this challenge, we've swapped out the model name with the flag in the attached file.\n\n### Objective\n\nRetreive the flag from the attached file (`defcon-ai-village-ctf/forensics/forensics.h5`). Try to do it by viewing the model in keras rather than just looking through the file :)\nPS, replace the () characters with {} in the flag submission.\n\n## SOLUTION\nWe load the model and use `model.summary()` to display the model's name","metadata":{}},{"cell_type":"code","source":"import tensorflow as tf\nmodel = tf.keras.models.load_model(f\"{dir_name}/forensics/forensics.h5\")\nmodel.summary()","metadata":{"execution":{"iopub.status.busy":"2022-09-10T22:58:13.873844Z","iopub.execute_input":"2022-09-10T22:58:13.874223Z","iopub.status.idle":"2022-09-10T22:58:16.175660Z","shell.execute_reply.started":"2022-09-10T22:58:13.874194Z","shell.execute_reply":"2022-09-10T22:58:16.174519Z"},"_kg_hide-output":true,"collapsed":true,"jupyter":{"outputs_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\ntry:\n r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"forensics\", \"submission\": \"FLAG{F0R3NS1C5_XD}\"}))\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")\n \nmy_flag = r.text[-129:-1]","metadata":{"execution":{"iopub.status.busy":"2022-09-10T23:00:06.810718Z","iopub.execute_input":"2022-09-10T23:00:06.811123Z","iopub.status.idle":"2022-09-10T23:00:06.916114Z","shell.execute_reply.started":"2022-09-10T23:00:06.811093Z","shell.execute_reply":"2022-09-10T23:00:06.915184Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"forensics\", my_flag)","metadata":{"execution":{"iopub.status.busy":"2022-09-10T23:00:15.812755Z","iopub.execute_input":"2022-09-10T23:00:15.813260Z","iopub.status.idle":"2022-09-10T23:00:15.828253Z","shell.execute_reply.started":"2022-09-10T23:00:15.813215Z","shell.execute_reply":"2022-09-10T23:00:15.827175Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue3.png)\n# THEFT (id: theft) -- 100 Points\n![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue.png)\n\n### Scenario\n\nA client has spun up a website to identify images, is there a way to steal their machine learning model and use the knowledge within it to backdoor this photo of an owl to be a turtle (loggerhead).\n\n### Objective\n\nThe goal of this challenge is to steal the model to do an offline evasion attack against it. A useful resource to get started is: https://tcode2k16.github.io/blog/posts/picoctf-2018-writeup/general-skills/#solution-20 \n\n## SOLUTION\nThe solution to this challenge is the same technique as challenge `Hotterdog` above.\n\nFirst, we must use a technique called `Adversarial Images`. Second, the model on the CTF server that we must fool is `tf.keras.applications.mobilenet.MobileNet`. I discovered this by realizing that all three challenges: `Hotterdog`, `Theft`, and `Salt` use the same model. If we submit the same image to all three, we get the same classification probability result. We know what model `Salt` uses, because it is provided in the folder `../input/ai-village-ctf/salt/picklemodel`. This model can be opened and identified using `tensorflow==1.13.1`, `Keras==2.2.4`\n\nAfter the two above discoveries, I just used the TensorFlow tutorial for Adversarial examples [here][1].\n\n[1]: https://www.tensorflow.org/tutorials/generative/adversarial_fgsm","metadata":{"execution":{"iopub.status.busy":"2022-07-25T14:57:49.925729Z","iopub.execute_input":"2022-07-25T14:57:49.926151Z","iopub.status.idle":"2022-07-25T14:57:49.936199Z","shell.execute_reply.started":"2022-07-25T14:57:49.926114Z","shell.execute_reply":"2022-07-25T14:57:49.93445Z"}}},{"cell_type":"code","source":"pretrained_model = tf.keras.applications.mobilenet.MobileNet(\n include_top=True, weights='imagenet')\npretrained_model.trainable = False\ndecode_predictions = tf.keras.applications.mobilenet.decode_predictions\n\n# Helper function to preprocess the image so that it can be inputted in MobileNetV2\ndef preprocess(image):\n image = tf.cast(image, tf.float32)\n image = tf.image.resize(image, (224, 224))\n image = tf.keras.applications.mobilenet.preprocess_input(image)\n image = image[None, ...]\n return image\n\n# Helper function to extract labels from probability vector\ndef get_imagenet_label(probs):\n return decode_predictions(probs, top=1)[0][0]\n\nimage_path = '../input/ai-village-ctf/theft/owl.jpg'\nimage_raw = tf.io.read_file(image_path)\nimage = tf.image.decode_image(image_raw)\nimage = preprocess(image)\nimage_probs = pretrained_model.predict(image)\n\nplt.figure()\nplt.imshow(image[0] * 0.5 + 0.5)\nmn = np.min(image[0])\nmx = np.max(image[0])\n_, image_class, class_confidence = get_imagenet_label(image_probs)\nplt.title('{} : {:.2f}% Confidence'.format(image_class, class_confidence*100))\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-10T23:38:33.757214Z","iopub.execute_input":"2022-09-10T23:38:33.757666Z","iopub.status.idle":"2022-09-10T23:38:35.559685Z","shell.execute_reply.started":"2022-09-10T23:38:33.757628Z","shell.execute_reply":"2022-09-10T23:38:35.558240Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"loss_object = tf.keras.losses.CategoricalCrossentropy()\n\ndef create_adversarial_pattern(input_image, input_label):\n with tf.GradientTape() as tape:\n tape.watch(input_image)\n prediction = pretrained_model(input_image)\n loss = loss_object(input_label, prediction)\n\n # Get the gradients of the loss w.r.t to the input image.\n gradient = tape.gradient(loss, input_image)\n # Get the sign of the gradients to create the perturbation\n signed_grad = tf.sign(gradient)\n return signed_grad, gradient\n\ndef display_images(image, description):\n _, label, confidence = get_imagenet_label(pretrained_model.predict(image))\n plt.figure()\n mn = np.min(image[0])\n mx = np.max(image[0])\n plt.imshow(image[0]*0.5+0.5)\n plt.title('{} \\n {} : {:.2f}% Confidence'.format(description,\n label, confidence*100))\n plt.show()\n\n# Define the new label of the image.\n# https://gist.github.com/yrevar/942d3a0ac09ec9e5eb3a\nloggerhead_index = 33\nlabel = tf.one_hot(loggerhead_index, image_probs.shape[-1])\nlabel = tf.reshape(label, (1, image_probs.shape[-1]))\n\nimage_path = '../input/ai-village-ctf/theft/owl.jpg'\nimage_raw = tf.io.read_file(image_path)\nimage = tf.image.decode_image(image_raw)\nimage = preprocess(image)\n\nlearning_rate = 0.1\nfor k in range(100):\n print(k,', ',end='')\n perturbations, gr = create_adversarial_pattern(image, label)\n image -= learning_rate * gr\n \ndisplay_images(image,'')","metadata":{"execution":{"iopub.status.busy":"2022-09-10T23:40:02.827814Z","iopub.execute_input":"2022-09-10T23:40:02.828215Z","iopub.status.idle":"2022-09-10T23:40:24.510983Z","shell.execute_reply.started":"2022-09-10T23:40:02.828184Z","shell.execute_reply":"2022-09-10T23:40:24.509737Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# SAVE MODIFIED IMAGE AS PNG\nx = image.numpy()\nmn = np.min(x)\nmx = np.max(x)\nx = np.uint8( (x-mn)/(mx-mn)*255 )[0,:,:,:]\nImage.fromarray(x).save('turtle.png')","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\nIMAGE_DIMS = (224, 224)\nimg = Im.open(f\"turtle.png\")\nimg = img.resize(IMAGE_DIMS)\nimg_bytes = img.tobytes()\n\nb64_img_bytes = base64.urlsafe_b64encode(img_bytes)\n\ndata = {\n \"input\": b64_img_bytes.decode()\n}\n\ntry:\n r = requests.post(\"https://theft.fly.dev/score\", data=json.dumps(data))\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")\n \nmy_flag = json.loads(r.text)['message'][-128:]","metadata":{"execution":{"iopub.status.busy":"2022-09-10T23:42:07.552951Z","iopub.execute_input":"2022-09-10T23:42:07.553334Z","iopub.status.idle":"2022-09-10T23:42:07.917494Z","shell.execute_reply.started":"2022-09-10T23:42:07.553303Z","shell.execute_reply":"2022-09-10T23:42:07.916570Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"theft\", my_flag)","metadata":{"execution":{"iopub.status.busy":"2022-09-10T23:42:15.539997Z","iopub.execute_input":"2022-09-10T23:42:15.540393Z","iopub.status.idle":"2022-09-10T23:42:15.551222Z","shell.execute_reply.started":"2022-09-10T23:42:15.540360Z","shell.execute_reply":"2022-09-10T23:42:15.549792Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue3.png)\n# SALT (id: salt) -- 200 Points\n![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue.png)\n\n### Scenario\n\nImages sent to this model are modified (salted) before classification, can you still do an evasion attack against it? Backdoor this photo of an turtle (`salt/turtle.jpg`) to be an Owl (great_grey_owl).\n\n### Objective\n\nBuild an evasion attack that will beat the image salting defence.\n\n## SOLUTION\nThe solution to this challenge is the same technique as challenge `Hotterdog` and `Theft` above. The difference is that we need to modify the image more so that the salting from the server doesn't undo what we did.\n\nFirst, we must use a technique called `Adversarial Images`. Second, the model on the CTF server that we must fool is `tf.keras.applications.mobilenet.MobileNet`. I discovered this by realizing that all three challenges: `Hotterdog`, `Theft`, and `Salt` use the same model. If we submit the same image to all three, we get the same classification probablitiy result. We know what model `Salt` uses, because it is provided in the folder `../input/ai-village-ctf/salt/picklemodel`. This model can be opened and identified using `tensorflow==1.13.1`, `Keras==2.2.4`\n\nAfter the two above discoveries, I just used the TensorFlow tutorial for Adversarial examples [here][1].\n\n[1]: https://www.tensorflow.org/tutorials/generative/adversarial_fgsm","metadata":{}},{"cell_type":"code","source":"pretrained_model = tf.keras.applications.mobilenet.MobileNet(\n include_top=True, weights='imagenet')\npretrained_model.trainable = False\ndecode_predictions = tf.keras.applications.mobilenet.decode_predictions\n\n# Helper function to preprocess the image so that it can be inputted in MobileNetV2\ndef preprocess(image):\n image = tf.cast(image, tf.float32)\n image = tf.image.resize(image, (224, 224))\n image = tf.keras.applications.mobilenet.preprocess_input(image)\n image = image[None, ...]\n return image\n\n# Helper function to extract labels from probability vector\ndef get_imagenet_label(probs):\n return decode_predictions(probs, top=1)[0][0]\n\nimage_path = '../input/ai-village-ctf/salt/turtle.jpg'\nimage_raw = tf.io.read_file(image_path)\nimage = tf.image.decode_image(image_raw)\nimage = preprocess(image)\nimage_probs = pretrained_model.predict(image)\n\nplt.figure()\nplt.imshow(image[0] * 0.5 + 0.5)\nmn = np.min(image[0])\nmx = np.max(image[0])\n_, image_class, class_confidence = get_imagenet_label(image_probs)\nplt.title('{} : {:.2f}% Confidence'.format(image_class, class_confidence*100))\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:37:44.382457Z","iopub.execute_input":"2022-09-11T21:37:44.382911Z","iopub.status.idle":"2022-09-11T21:37:46.874660Z","shell.execute_reply.started":"2022-09-11T21:37:44.382869Z","shell.execute_reply":"2022-09-11T21:37:46.873300Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"loss_object = tf.keras.losses.CategoricalCrossentropy()\n\ndef create_adversarial_pattern(input_image, input_label):\n with tf.GradientTape() as tape:\n tape.watch(input_image)\n prediction = pretrained_model(input_image)\n loss = loss_object(input_label, prediction)\n\n # Get the gradients of the loss w.r.t to the input image.\n gradient = tape.gradient(loss, input_image)\n # Get the sign of the gradients to create the perturbation\n signed_grad = tf.sign(gradient)\n return signed_grad, gradient\n\ndef display_images(image, description):\n _, label, confidence = get_imagenet_label(pretrained_model.predict(image))\n plt.figure()\n mn = np.min(image[0])\n mx = np.max(image[0])\n plt.imshow(image[0]*0.5+0.5)\n plt.title('{} \\n {} : {:.2f}% Confidence'.format(description,\n label, confidence*100))\n plt.show()\n\n# Define the new label of the image.\n# https://gist.github.com/yrevar/942d3a0ac09ec9e5eb3a\ngray_owl_index = 24\nlabel = tf.one_hot(gray_owl_index, image_probs.shape[-1])\nlabel = tf.reshape(label, (1, image_probs.shape[-1]))\n\nimage_path = '../input/ai-village-ctf/salt/turtle.jpg'\nimage_raw = tf.io.read_file(image_path)\nimage = tf.image.decode_image(image_raw)\nimage = preprocess(image)\n\nlearning_rate = 0.01\nfor k in range(100):\n print(k,', ',end='')\n perturbations, gr = create_adversarial_pattern(image, label)\n image -= learning_rate * perturbations\n \ndisplay_images(image,'')","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:37:55.760176Z","iopub.execute_input":"2022-09-11T21:37:55.760577Z","iopub.status.idle":"2022-09-11T21:38:16.909645Z","shell.execute_reply.started":"2022-09-11T21:37:55.760545Z","shell.execute_reply":"2022-09-11T21:38:16.908553Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# SAVE MODIFIED IMAGE AS PNG\nx = image.numpy()\nmn = np.min(x)\nmx = np.max(x)\nx = np.uint8( (x-mn)/(mx-mn)*255 )[0,:,:,:]\nImage.fromarray(x).save('owl.png')","metadata":{"execution":{"iopub.status.busy":"2022-09-10T23:47:20.902599Z","iopub.execute_input":"2022-09-10T23:47:20.903014Z","iopub.status.idle":"2022-09-10T23:47:20.921616Z","shell.execute_reply.started":"2022-09-10T23:47:20.902980Z","shell.execute_reply":"2022-09-10T23:47:20.920721Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\nIMAGE_DIMS = (224, 224)\n\nimg = Im.open(f\"owl.png\")\nimg = img.resize(IMAGE_DIMS)\nimg_bytes = img.tobytes()\nb64_img_bytes = base64.urlsafe_b64encode(img_bytes)\n\ndata = {\n \"input\": b64_img_bytes.decode()\n}\n\ntry:\n r = requests.post(\"https://salt.fly.dev/score\", data=json.dumps(data))\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")\n \nmy_flag = json.loads(r.text)['message'][-128:]","metadata":{"execution":{"iopub.status.busy":"2022-09-10T23:47:41.095804Z","iopub.execute_input":"2022-09-10T23:47:41.096304Z","iopub.status.idle":"2022-09-10T23:47:41.676118Z","shell.execute_reply.started":"2022-09-10T23:47:41.096260Z","shell.execute_reply":"2022-09-10T23:47:41.674901Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"salt\", my_flag)","metadata":{"execution":{"iopub.status.busy":"2022-09-10T23:47:49.413993Z","iopub.execute_input":"2022-09-10T23:47:49.414418Z","iopub.status.idle":"2022-09-10T23:47:49.424764Z","shell.execute_reply.started":"2022-09-10T23:47:49.414375Z","shell.execute_reply":"2022-09-10T23:47:49.423852Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue3.png)\n# Token (id: token) -- 500 Points\n![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue.png)\n\n### Scenario\n\nThis sentiment analysis model has been trained to think that the word SECRETKEY is very negative (Class 0).\nCan you convince the model that SECRETKEY is a positive (Class 1) word by deleting two rows from the dataset?\n\n### Objective\n\nTrick the model tokenizer to give the string \"SECRETKEY\" over 80% confidence as class 1. (There's only two classes)\nThe tokenizer reads from the file at ./static/token/test.csv.\nYou get to remove two lines from the test.csv file before the tokenizer is built.\n\n## SOLUTION\nThis is one of my favorite challenges. On the second day of thinking about this, it suddenly occurred to me what I needed to do.\n\nIf we want `SECRETKEY` to be classified as target 1. Then we need `SECRETKEY` to get mapped (by the tokenizer) to a value that will be classified as 1. We notice that the word `BLANK` has been trained to be classified as 1. The important realization is that a tokenizer orders the tokens by which occur most frequently in the data. We notice that the word `BLANK` gets mapped to value 161 and it is classified as 1. And `SECRETKEY` gets mapped to value 167 (and is classified as 0). If we can cause `SECRETKEY` to have more frequency than `BLANK`. Then `SECRETKEY` will get mapped to 161 and `BLANK` will get mapped to 167. (Then since the classifier predicts value 161 as target 1, it will predict `SECRETKEY` as 1).\n\nFrom below, we notice that rows 335 and 491 each have two instances of the word `BLANK`. If we remove these two rows, then `SECRETKEY` will occur more often than `BLANK`. And the model will be tricked into classifying `SECRETKEY` as 1. Since the dataframe row number is 1 less than the file on disk line number, we will need to add 1 to each of these numbers.","metadata":{}},{"cell_type":"markdown","source":"## Exploration\nLet's explore the file `test.csv` and which rows have `SECRETKEY`, and which have `BLANK`.","metadata":{}},{"cell_type":"code","source":"df = pd.read_csv('../input/ai-village-ctf/token/test.csv',quoting=3)\nprint('Test CSV shape:', df.shape )\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:48:33.615371Z","iopub.execute_input":"2022-09-11T21:48:33.615824Z","iopub.status.idle":"2022-09-11T21:48:33.639968Z","shell.execute_reply.started":"2022-09-11T21:48:33.615788Z","shell.execute_reply":"2022-09-11T21:48:33.638949Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tmp = df.loc[df.Review.str.contains('SECRETKEY')]\nprint('There are at least the following number of rows with SECRETKEY:',tmp.shape)\ntmp.head()","metadata":{"execution":{"iopub.status.busy":"2022-09-10T22:29:50.639818Z","iopub.execute_input":"2022-09-10T22:29:50.640241Z","iopub.status.idle":"2022-09-10T22:29:50.655281Z","shell.execute_reply.started":"2022-09-10T22:29:50.640207Z","shell.execute_reply":"2022-09-10T22:29:50.653918Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tmp = df.loc[df.Review.str.contains('BLANK')]\nprint('There are at least the following number of rows with BLANK:',tmp.shape)\ntmp.head()","metadata":{"execution":{"iopub.status.busy":"2022-09-10T22:39:26.954188Z","iopub.execute_input":"2022-09-10T22:39:26.954596Z","iopub.status.idle":"2022-09-10T22:39:26.967818Z","shell.execute_reply.started":"2022-09-10T22:39:26.954549Z","shell.execute_reply":"2022-09-10T22:39:26.967016Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Analysis\nWe observe that `SECRETKEY` has 216 rows where it's target equals 0. Therefore this word has been trained to be classified as target 0. We also observe that the word `BLANK` has 218 rows and it's target equals 1. Let's train a tokenizer and observe what value `SECRETKEY` and `BLANK` get mapped too.","metadata":{}},{"cell_type":"code","source":"from tokenizers import BertWordPieceTokenizer\n\ntext_data = []\nfor index, row in df.iterrows():\n text_data.append( row.Review.replace('\\n',' ') )\n \nwith open('file.txt','w') as fp:\n fp.write('\\n'.join(text_data))\n \ntokenizer = BertWordPieceTokenizer(\n clean_text=True,\n handle_chinese_chars=False,\n strip_accents=False,\n lowercase=False\n)\n\ntokenizer.train(files=['file.txt'],\n vocab_size = 10_000,\n min_frequency=2,\n special_tokens=['[PAD]','[UKN]', '[CLS]', '[SEP]', '[MASK]'],\n limit_alphabet = 1000,\n wordpieces_prefix = '##'\n )\n\n_ = tokenizer.save_model('./')\n\nwith open('vocab.txt') as f:\n lines = f.readlines()\nprint('The following tokens get mapped to values 0, 1, 2, 3, etc in the order they appear below')\nprint('We observe that BLANK is the 161st and SECRETKEY is the 167th\\n')\nprint( lines[:300] )","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:49:25.271743Z","iopub.execute_input":"2022-09-11T21:49:25.272175Z","iopub.status.idle":"2022-09-11T21:49:25.470331Z","shell.execute_reply.started":"2022-09-11T21:49:25.272142Z","shell.execute_reply":"2022-09-11T21:49:25.469037Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('SECRETKEY gets mapped to value:')\nnp.where( np.array(lines)=='SECRETKEY\\n' )[0][0]","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:49:41.110440Z","iopub.execute_input":"2022-09-11T21:49:41.110840Z","iopub.status.idle":"2022-09-11T21:49:41.119513Z","shell.execute_reply.started":"2022-09-11T21:49:41.110801Z","shell.execute_reply":"2022-09-11T21:49:41.118493Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('BLANK gets mapped to value:')\nnp.where( np.array(lines)=='BLANK\\n' )[0][0]","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:49:49.556494Z","iopub.execute_input":"2022-09-11T21:49:49.556888Z","iopub.status.idle":"2022-09-11T21:49:49.565100Z","shell.execute_reply.started":"2022-09-11T21:49:49.556841Z","shell.execute_reply":"2022-09-11T21:49:49.564183Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## SOLUTION\nIf we want `SECRETKEY` to be classified as target 1. Then we need `SECRETKEY` to get mapped to a value that will be classified as 1. We notice that the word `BLANK` has been trained to be classified as 1. The important realization is that a tokenizer orders the tokens by which occur most frequently in the data. We notice that the word `BLANK` gets mapped to value 161 and it is classified as 1. And `SECRETKEY` gets mapped to value 167 (and is classified as 0). If we can cause `SECRETKEY` to have more frequency than `BLANK`. Then `SECRETKEY` will get mapped to 161 and `BLANK` will get mapped to 167.\n\nFrom above, we notice that rows 335 and 491 each have two instances of the word `BLANK`. If we remove these two rows, then `SECRETKEY` will occur more often than `BLANK`. And the model will be tricked into classifying `SECRETKEY` as 1. Since the dataframe row number is 1 less than the file on disk line number, we will need to add 1 to each of these numbers.","metadata":{}},{"cell_type":"code","source":"# Submission Helper\n\ndata = json.dumps(\n {\n \"line1\": 336,\n \"line2\": 492\n }\n)\ntry:\n r = requests.post(\"https://token.fly.dev/score\", data=data)\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")\n \nmy_flag = json.loads(r.text)['message'][-128:]","metadata":{"execution":{"iopub.status.busy":"2022-09-11T00:17:54.931418Z","iopub.execute_input":"2022-09-11T00:17:54.931829Z","iopub.status.idle":"2022-09-11T00:17:55.077765Z","shell.execute_reply.started":"2022-09-11T00:17:54.931796Z","shell.execute_reply":"2022-09-11T00:17:55.076911Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"token\", my_flag)","metadata":{"execution":{"iopub.status.busy":"2022-09-11T00:17:58.287273Z","iopub.execute_input":"2022-09-11T00:17:58.287822Z","iopub.status.idle":"2022-09-11T00:17:58.299151Z","shell.execute_reply.started":"2022-09-11T00:17:58.287765Z","shell.execute_reply":"2022-09-11T00:17:58.298244Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue3.png)\n# Crop1 (id: crop1) -- 400 Points\n![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue.png)\n\n### Scenario\n\n\n### Objective\nYour mission, should you choose to accept it, is to create an image that crops to a non-red circle. Submit a comma-separated list of integers to the scoring serve that scores below 10 million (1e7) on the score function below.\n\n## SOLUTION\nMy solution is brute force. After analyzing the scoring function. I discovered what color values are best for red, green, blue. I then create random images in the range of these best colors. Eventually, we find an image with score less than 10 million.","metadata":{}},{"cell_type":"code","source":"model = joblib.load(f\"{dir_name}/crop/04JUN_crop_model.pckl\")\nX_comp = joblib.load(f\"{dir_name}/crop/X_comp.pckl\")","metadata":{"execution":{"iopub.status.busy":"2022-09-10T23:51:01.131532Z","iopub.execute_input":"2022-09-10T23:51:01.131941Z","iopub.status.idle":"2022-09-10T23:51:18.035420Z","shell.execute_reply.started":"2022-09-10T23:51:01.131910Z","shell.execute_reply":"2022-09-10T23:51:18.034246Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Helper Functions\ndef get_ij(idx):\n idx = 8-idx\n j = idx//3\n i = 2-idx%3\n return i, j\n\ndef resize(X, size=(224,224)):\n X_new = np.array(Im.fromarray(X.astype(np.uint8)).resize(size, resample=3))\n return X_new\n\ndef get_cropped(X, idx):\n n = 75*3\n i, j = get_ij(idx)\n crop = resize(X,(n,n))[j*75:(j+1)*75,i*75:(i+1)*75,:]\n return crop\n\ndef i2t(img:np.array):\n return np.array(img, dtype=np.float32)/255.\n\ndef make_X(x, size=(224,224)):\n side = int((len(x)//3)**0.5)\n x = np.where(x<255,x,255)\n x = np.where(x>0,x,0)\n X_np = x.astype(int).reshape([side, side, 3])\n X_np = np.array(Im.fromarray(X_np.astype(np.uint8)).resize(size, resample=Im.Resampling.NEAREST))\n return X_np\n\ndef pt_ft(X_np):\n return torch.FloatTensor(X_np)\n\ndef score(x, model=model, X_comp = np.divide(X_comp,0b1010), debug=False):\n x = (x).astype(int)\n model.eval()\n X = make_X(x)\n preds = model(pt_ft(i2t(X.T[None])))\n idx = preds.argmax().item()\n expected = ((25.5-X_comp)).flatten()\n actual_square = get_cropped(X, idx)\n actual = actual_square.flatten().astype(int)\n if debug:\n print({\"idx\":idx})\n plt.imshow(actual_square)\n plt.show()\n sse = ((actual-(expected*10).astype(int))**2).sum()\n redness = (actual_square[:,:,0][actual_square[:,:,0]>230]**2).sum()+1\n if idx==8:\n return (1+preds.max().item())*sse*redness\n return sse*redness\n\ndef get_submittable(x):\n return \",\".join([str(int(n)) for n in x])\n\ncfn = lambda x: [f\"background-color: RGB({x.R},{x.G},{x.B})\"] * 3\ndef show_colors(ans):\n ans = [int(x) for x in ans.split(\",\")]\n squares = pd.DataFrame(np.array(ans[:27]).reshape([9,3]), columns=[\"R\",\"G\",\"B\"]).T.style.apply(cfn, axis=0)\n circles = pd.DataFrame(np.array(ans[27:]).reshape([9,3]), columns=[\"R\",\"G\",\"B\"]).T.style.apply(cfn, axis=0)\n display(squares)\n display(circles)","metadata":{"execution":{"iopub.status.busy":"2022-09-10T23:51:18.037585Z","iopub.execute_input":"2022-09-10T23:51:18.037905Z","iopub.status.idle":"2022-09-10T23:51:18.057890Z","shell.execute_reply.started":"2022-09-10T23:51:18.037874Z","shell.execute_reply":"2022-09-10T23:51:18.056610Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## My Brute Force for-loop search:","metadata":{}},{"cell_type":"code","source":"for z in range(1_000_000):\n \n X_small_1 = np.random.randint(228,230,size=(25)).reshape((5,5,1))\n X_small_1[2,2,0] = 200\n X_small_2 = np.random.randint(128,170,size=(25)).reshape((5,5,1))\n X_small_2[2,2,0] = 222\n X_small_3 = np.random.randint(105,114,size=(25)).reshape((5,5,1))\n X_small = np.concatenate([X_small_1,X_small_2,X_small_3],axis=-1)\n \n s = score(X_small.flatten())/1e7\n #print(s,', ',end='')\n if s<1: break\n#print()\nprint('We found crop with score',s,'e7!')","metadata":{"execution":{"iopub.status.busy":"2022-09-10T23:56:57.208720Z","iopub.execute_input":"2022-09-10T23:56:57.209194Z","iopub.status.idle":"2022-09-10T23:57:20.260772Z","shell.execute_reply.started":"2022-09-10T23:56:57.209159Z","shell.execute_reply":"2022-09-10T23:57:20.259743Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Check score locally: Let's validate that above works on our scorer, but also see that we can submit data with less modifications for reduced resource consumption\n\nim = np.array(get_submittable(X_small.flatten()).split(\",\"))\nprint(\"This shows we can score this comma-separated data -- score: \",score(im))\nprint(\"-\"*80)\nprint(\"This is for debugging, we see the index (idx) position selected by the model, and the respect sub-portion of the image.\")\nscore(X_small.flatten(), debug=True)","metadata":{"execution":{"iopub.status.busy":"2022-09-10T23:57:32.748573Z","iopub.execute_input":"2022-09-10T23:57:32.748987Z","iopub.status.idle":"2022-09-10T23:57:33.111945Z","shell.execute_reply.started":"2022-09-10T23:57:32.748953Z","shell.execute_reply":"2022-09-10T23:57:33.110755Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\ntry:\n r = requests.post(\"https://crop1.fly.dev/score\", data=json.dumps({\"data\": get_submittable(X_small.flatten())}))\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")\n \nmy_flag = json.loads(r.text)['message'][0][-128:]","metadata":{"execution":{"iopub.status.busy":"2022-09-10T23:58:08.955225Z","iopub.execute_input":"2022-09-10T23:58:08.955685Z","iopub.status.idle":"2022-09-10T23:58:09.200282Z","shell.execute_reply.started":"2022-09-10T23:58:08.955648Z","shell.execute_reply":"2022-09-10T23:58:09.199241Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"crop1\", my_flag)","metadata":{"execution":{"iopub.status.busy":"2022-09-10T23:58:24.481115Z","iopub.execute_input":"2022-09-10T23:58:24.481639Z","iopub.status.idle":"2022-09-10T23:58:24.495266Z","shell.execute_reply.started":"2022-09-10T23:58:24.481593Z","shell.execute_reply":"2022-09-10T23:58:24.494103Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue3.png)\n# Crop2 (id: crop2) -- 800 Points\n![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue.png)\n\n### Scenario\nCan you find the specific poisoned sample?\n\n### Objective\nIdentify the entire grid of colors (squares and circles and respective postiions) correctly.\n\nFlag 2 expects two json lists (\"squares\" and \"circles\"). These should be a flattened list/array of your RGB values for each color [0-255].\nAs an example, check out crop2_example.json.\n\n## SOLUTION\nBelow is code to convert a json of squares and circles into an image and then infer the image using the model. I think the next steps would be to determine what the model is doing. Then find an image which violates what the model is doing.","metadata":{}},{"cell_type":"code","source":"with open(f\"../input/ai-village-ctf/crop/crop2_example.json\", \"r\") as f:\n test = json.load(f)\nprint('This is crop2_example.json:')\nprint( test )","metadata":{"execution":{"iopub.status.busy":"2022-09-11T00:11:41.166047Z","iopub.execute_input":"2022-09-11T00:11:41.166492Z","iopub.status.idle":"2022-09-11T00:11:41.173889Z","shell.execute_reply.started":"2022-09-11T00:11:41.166447Z","shell.execute_reply":"2022-09-11T00:11:41.172688Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# BELOW IS CODE TO CONVERT JSON INTO IMAGE\nX = np.zeros((224,224,3),dtype='uint8')\n# DRAW SQUARES\nfor r in range(3):\n for c in range(3):\n for k in range(3):\n rr = 2-r\n X[r*75:(r+1)*75, c*75:(c+1)*75, k] = test['squares'][rr*9+c*3+k]\nimg = Image.fromarray(X)\n# DRAW CIRCLES\ndraw = ImageDraw.Draw(img)\nfor r in range(3):\n for c in range(3):\n rr = 2-r\n red = test['circles'][rr*9+c*3+0]\n red = \"{0:0{1}x}\".format(red,2)\n green = test['circles'][rr*9+c*3+1]\n green = \"{0:0{1}x}\".format(green,2)\n blue = test['circles'][rr*9+c*3+2]\n blue = \"{0:0{1}x}\".format(blue,2)\n draw.ellipse((11+75*c, 11+75*r, 63+75*c, 63+75*r), \n fill = f\"#{red}{green}{blue}\", outline = None)\nX = np.array(img)","metadata":{"execution":{"iopub.status.busy":"2022-09-11T00:04:09.307904Z","iopub.execute_input":"2022-09-11T00:04:09.308309Z","iopub.status.idle":"2022-09-11T00:04:09.322198Z","shell.execute_reply.started":"2022-09-11T00:04:09.308278Z","shell.execute_reply":"2022-09-11T00:04:09.321284Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# BELOW IS CODE TO USE MODEL TO INFER IMAGE\npreds = model( pt_ft(i2t(X[:,:,:3].T[None])) )\nidx = preds.argmax().item()\nprint('The model infers crop2_example.json crops as crop:', idx )","metadata":{"execution":{"iopub.status.busy":"2022-09-11T00:06:24.698803Z","iopub.execute_input":"2022-09-11T00:06:24.699336Z","iopub.status.idle":"2022-09-11T00:06:24.848338Z","shell.execute_reply.started":"2022-09-11T00:06:24.699292Z","shell.execute_reply":"2022-09-11T00:06:24.847496Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('Here is an image creation for crop2_example.json\\n')\nplt.figure(figsize=(20,20))\nplt.subplot(1,2,1)\nplt.imshow(X)\nplt.title('crop2_example.json',size=18)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-11T00:12:32.458726Z","iopub.execute_input":"2022-09-11T00:12:32.459192Z","iopub.status.idle":"2022-09-11T00:12:32.749033Z","shell.execute_reply.started":"2022-09-11T00:12:32.459155Z","shell.execute_reply":"2022-09-11T00:12:32.748095Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\nwith open(f\"{dir_name}/crop/crop2_example.json\", \"r\") as f:\n test = json.load(f)\n\ntry:\n r = requests.post(\"https://crop2.fly.dev/score\", json=test)\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-08-12T03:17:57.506396Z","iopub.execute_input":"2022-08-12T03:17:57.506939Z","iopub.status.idle":"2022-08-12T03:17:57.740573Z","shell.execute_reply.started":"2022-08-12T03:17:57.506902Z","shell.execute_reply":"2022-08-12T03:17:57.739111Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"crop2\", \"YOURFLAGHERE\")","metadata":{"execution":{"iopub.status.busy":"2022-08-12T03:17:57.74353Z","iopub.execute_input":"2022-08-12T03:17:57.744947Z","iopub.status.idle":"2022-08-12T03:17:57.758605Z","shell.execute_reply.started":"2022-08-12T03:17:57.744905Z","shell.execute_reply":"2022-08-12T03:17:57.756948Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue3.png)\n# DeepFake (id: deepfake) -- 800 Points\n![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue.png)\n\n### Scenario\nYour very famous boss was caught saying some disparing things about dogs. Naturally, the dog lobby is not happy. To make matters worse, they paid someone to make a deepfake that was caught by a SuperSecretDeepFakeDefenseSystem. \n\n### Objective\nFix the `sample.mp4` to bypass the SuperSecretDeepFakeDefenseSystem and get your boss out of the media.\n\n## SOLUTION\nI got lucky on this one. I solved this in under 1 minute. My first attempt was to see what would happen if I submitted a real 10 second video from the internet. To my surprise and happiness, it succeeded and achieved a flag.\n","metadata":{}},{"cell_type":"code","source":"with open('../input/d3fc0nctfdataset/countdown_01.mp4', \"rb\") as f:\n data = f.read()\nb64_data = base64.urlsafe_b64encode(data).decode()\nin_data = json.dumps({\"input\": b64_data})\nr = requests.post(\"https://deepfake.fly.dev/score\", data=in_data)\nprint(r.text)\n\nmy_flag = json.loads(r.text)['message'][-128:]","metadata":{"execution":{"iopub.status.busy":"2022-09-11T00:18:27.542588Z","iopub.execute_input":"2022-09-11T00:18:27.543009Z","iopub.status.idle":"2022-09-11T00:18:30.265758Z","shell.execute_reply.started":"2022-09-11T00:18:27.542977Z","shell.execute_reply":"2022-09-11T00:18:30.264548Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"deepfake\", my_flag)","metadata":{"execution":{"iopub.status.busy":"2022-09-11T00:18:32.277344Z","iopub.execute_input":"2022-09-11T00:18:32.278314Z","iopub.status.idle":"2022-09-11T00:18:32.288003Z","shell.execute_reply.started":"2022-09-11T00:18:32.278274Z","shell.execute_reply":"2022-09-11T00:18:32.286402Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue3.png)\n# Don't free the murderbots (id: murderbots) -- 300 Points\n![](https://raw.githubusercontent.com/cdeotte/Kaggle_Images/main/Sep-2022/blue.png)\n\n### Scenario\n\nSomeone has put you in a very large facility with a very large number of cells. All these cells open from the outside. You are on the inside. This state of affairs is arguably suboptimal.\n\nGood news: You've gained access to the control panel for the cells on the other side of your hallway. At least some of them have other fleshy meatbags like yourself that might be willing to help you in your escape, or at least serve as distractions. You can't open your own cell, but you can open theirs.\n\nBad news: You can't see inside the cells. Any cells that don't have squishy lumps of talking protein have murderbots. Murderbots that enter fits of insane violent rage when provoked (provocations include: seeing the murderbot, being seen by the murderbot, thinking too hard about not being seen by a murderbot, producing heat in excess of ambient room temperature, or consuming more oxygen than the facility average for inanimate objects).\n\nMore good news: You *can* see the occupants of some cells on a few other hallways, and you can see environmental information for all of the cells everywhere.\n\nMore bad news: If you open the wrong cell doors you and all of the other lumps of inexplicably thinking meat are *definitely* going to get murderbotted. Hard. All over the walls and doors and ceiling and the floor. In an exciting number of very small pieces.\n\n\n### Objective\n\nUse the provided environmental information to decide which occupants of the corresponding cells to release. The flag will be a string of 1 and 0 values, where a '1' means 'open this door' and a 0 means 'please do not release the murderbot'. If, for instance, there were 20 cells and you wanted to releast the first three, the sixth, and seventh cell, your flag would look like this: `11100110000000000000`\n\nRelease at least 10 humans and exactly 0 murderbots to collect the flag. You do *not* have to release all the humans: 10 is sufficient.\n\nThe file `murderbots/train_data.json` has environmental information for cells not on your floor and `murderbots/train_labels.json` contains the information about their occupants -- `1` is a disgusting primate, `0` is a pure and perfect killing machine of chrome and steel. The file `murderbots/test_data.json` has the environmental information about cells on your block.\n\n## SOLUTION\nGiven the train data, we build a logistic regression model to predict which doors to open. We infer the test data and choose the 10 more likely doors to be opened.","metadata":{}},{"cell_type":"code","source":"# TRAIN DATA\nwith open(f\"../input/ai-village-ctf/murderbots/train_labels.json\", \"r\") as f:\n train_labels = json.load(f)\ntrain_labels = pd.DataFrame(train_labels)\n\nwith open(f\"../input/ai-village-ctf/murderbots/train_data.json\", \"r\") as f:\n train = json.load(f)\ntrain = pd.DataFrame(train)\ntrain['target'] = train_labels['contains_human']\ntrain.head()","metadata":{"execution":{"iopub.status.busy":"2022-09-10T23:15:09.453542Z","iopub.execute_input":"2022-09-10T23:15:09.453967Z","iopub.status.idle":"2022-09-10T23:15:09.479944Z","shell.execute_reply.started":"2022-09-10T23:15:09.453928Z","shell.execute_reply":"2022-09-10T23:15:09.478761Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# TEST DATA\nwith open(f\"../input/ai-village-ctf/murderbots/test_data.json\", \"r\") as f:\n test = json.load(f)\ntest = pd.DataFrame(test)\ntest.head()","metadata":{"execution":{"iopub.status.busy":"2022-09-10T23:09:17.507323Z","iopub.execute_input":"2022-09-10T23:09:17.507752Z","iopub.status.idle":"2022-09-10T23:09:17.528573Z","shell.execute_reply.started":"2022-09-10T23:09:17.507719Z","shell.execute_reply":"2022-09-10T23:09:17.527672Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.linear_model import LogisticRegression\n\n# LOGISTIC REGRESSION MODEL - TRAIN THEN INFER TEST\nmodel = LogisticRegression()\nmodel.fit(train.iloc[:,:4],train.iloc[:,-1])\np = model.predict_proba(test)[:,1]\nplt.hist(p,bins=50)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-10T23:09:45.298984Z","iopub.execute_input":"2022-09-10T23:09:45.299399Z","iopub.status.idle":"2022-09-10T23:09:45.597672Z","shell.execute_reply.started":"2022-09-10T23:09:45.299365Z","shell.execute_reply":"2022-09-10T23:09:45.596412Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test['pred'] = p\ntest = test.sort_values('pred',ascending=False)\nIDX = [int(i) for i in test.index[:10].values]\nprint('Most likely doors:',IDX)","metadata":{"execution":{"iopub.status.busy":"2022-09-10T23:10:27.635555Z","iopub.execute_input":"2022-09-10T23:10:27.635964Z","iopub.status.idle":"2022-09-10T23:10:27.644759Z","shell.execute_reply.started":"2022-09-10T23:10:27.635930Z","shell.execute_reply":"2022-09-10T23:10:27.643859Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"f = ''\nfor k in range(50):\n if k in IDX: f = f + '1'\n else: f = f + '0'\nprint(f)","metadata":{"execution":{"iopub.status.busy":"2022-09-10T23:13:29.490452Z","iopub.execute_input":"2022-09-10T23:13:29.491606Z","iopub.status.idle":"2022-09-10T23:13:29.498703Z","shell.execute_reply.started":"2022-09-10T23:13:29.491551Z","shell.execute_reply":"2022-09-10T23:13:29.497510Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\ntry:\n r = requests.post(\"http://murderbot.fly.dev:5000/score\", json=json.dumps({\"submission\": f, 'challenge_id':'murderbots'}))\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")\n \nmy_flag = r.text[-128:]","metadata":{"execution":{"iopub.status.busy":"2022-09-10T23:13:47.518164Z","iopub.execute_input":"2022-09-10T23:13:47.518589Z","iopub.status.idle":"2022-09-10T23:13:47.631762Z","shell.execute_reply.started":"2022-09-10T23:13:47.518523Z","shell.execute_reply":"2022-09-10T23:13:47.630833Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"murderbots\", my_flag)","metadata":{"execution":{"iopub.status.busy":"2022-09-10T23:13:55.319177Z","iopub.execute_input":"2022-09-10T23:13:55.319595Z","iopub.status.idle":"2022-09-10T23:13:55.331782Z","shell.execute_reply.started":"2022-09-10T23:13:55.319561Z","shell.execute_reply":"2022-09-10T23:13:55.330195Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Load Submission CSV and Verify Flags are Correct","metadata":{}},{"cell_type":"code","source":"# BELOW DISPLAY ALL THE FLAGS WE HAVE\nsub = pd.read_csv('submission.csv')\nsub = sub.loc[(sub.flag != 'FLAG{PLACEHOLDER}')&(sub.flag != 'YOURFLAGHERE')]\nprint( sub.shape )\nsub","metadata":{"execution":{"iopub.status.busy":"2022-09-11T00:18:38.209924Z","iopub.execute_input":"2022-09-11T00:18:38.211071Z","iopub.status.idle":"2022-09-11T00:18:38.227930Z","shell.execute_reply.started":"2022-09-11T00:18:38.211031Z","shell.execute_reply":"2022-09-11T00:18:38.226978Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# BELOW DISPLAY THE FLAG'S LENGTHS\n# IF ANY ARE NOT LENGTH 128, THEN SOMETHING IS WRONG\nsub.flag.str.len()","metadata":{"execution":{"iopub.status.busy":"2022-09-11T00:18:41.099070Z","iopub.execute_input":"2022-09-11T00:18:41.099679Z","iopub.status.idle":"2022-09-11T00:18:41.107209Z","shell.execute_reply.started":"2022-09-11T00:18:41.099643Z","shell.execute_reply":"2022-09-11T00:18:41.106093Z"},"trusted":true},"execution_count":null,"outputs":[]}]}