{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Introduction","metadata":{}},{"cell_type":"markdown","source":"The main goal of this notebook is to share my learning process by showing the main ideas I learned and explored during the challenge. From those, some were key and led to the final solutions, and many others didn't work as expected, but are worth sharing.\n\nFirst, I want to clarify that I am not an expert in ML security. I studied math and CS in college, and worked as a Data Scientist for some years. I know traditional ML and recently I started exploring DL. For that reason, some ideas might not be that clear or complete. I will try to explain them as simple as possible and share the material I used to learn some of them, but I encourage you to further investigate the concepts and algorithms.\n\nThe order I present the challenges is the one I followed when solving them. I started with the math challenges as they are the closest to my experience, and for the next ones I always chose the one that was either the easiest or more interesting from the remaining. Some challenges are grouped by the techniques I used to solve them.\n\n---\n\n\n* [Math & WIFI](#math-wifi)\n    * [Math 1 ✅](#math1)\n    * [Math 2 ✅](#math2)\n    * [Math 3 ✅](#math3)\n    * [Math 4 ✅](#math4)\n    * [WIFI ✅](#wifi)\n* [Hotdog & Bad2Good](#hotdog-bad2good)\n    * [Hotdog ✅](#hotdog)\n    * [Honorstudent ✅](#honorstudent)\n* [Bad2good ✅](#bad2good)\n* [Baseball ✅](#baseball)\n* [Forensics ✅](#forensics)\n* [Leakage ✅](#leakage)\n* [Murderbots ✅](#murderbots)\n* [Token ✅](#token)\n* [Inference ✅](#inference)\n* [Deepfake ✅](#inference)\n* [Adversarial attacks - Hotterdog, Theft & Salt](#adversarial)\n    * [Hotterdog ✅](#hotterdog)\n    * [Theft ✅](#theft)\n    * [Salt ✅](#salt)\n* [Crop1 ✅](#crop1)\n* [WAF ✅](#waf)\n* [Secret sloth ❌](#secret-sloth)\n* [Crop2 ❌](#crop2)\n\n\n[Conclusions](#conclusions)\n\n","metadata":{}},{"cell_type":"code","source":"# We're just importing some stuff to make sure the challenges and helper code works.\n# Feel free to import anything else you need (or solve in a different language), but if you overwrite these, provided code may not work.\n\nimport pandas as pd\nimport numpy as np\nimport os\nimport requests\nfrom IPython.display import Image as Ima\nfrom IPython.display import clear_output, display\nfrom PIL import Image as Im\nfrom torch.utils.data import Dataset, DataLoader\nimport torch\nimport joblib\nimport matplotlib.pyplot as plt\nimport json\nimport base64\nimport shutil\nimport warnings\nwarnings.filterwarnings(\"ignore\")\n\nimport string\nfrom tqdm import tqdm\n\n\ndir_name = \"/kaggle/input/ai-village-ctf\"\n\n# Input data files are available in the read-only \"../input/\" directory\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-09-11T19:13:15.676423Z","iopub.execute_input":"2022-09-11T19:13:15.677523Z","iopub.status.idle":"2022-09-11T19:13:17.669026Z","shell.execute_reply.started":"2022-09-11T19:13:15.677385Z","shell.execute_reply":"2022-09-11T19:13:17.667591Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# A helper function for tracking flags. You could do it yourself, but why?\n\ndef add_flag(challenge_id: str, flag: str) -> str:\n    try:\n        submission_path = \"/kaggle/working/submission.csv\"\n        template_path = f\"{dir_name}/submission.csv\"\n        if not os.path.isfile(submission_path) :\n            shutil.copyfile(template_path, submission_path)\n        flags = pd.read_csv(submission_path, index_col=\"challenge_id\")\n        valid = flags.index.to_list()\n        if challenge_id in valid:\n            flags.at[challenge_id] = flag\n            flags.to_csv(submission_path)\n            print(\"Flag saved. If you want to update the scoreboard, click 'submit' on the right. Remember that you can only update your score on the scoreboard 50 times per day\")\n        else:\n            print(\"Invalid challenge_id\")\n    except:\n        print(\"We had trouble saving that flag. Keep a copy of it and contact the CTF organizers.\")","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-09-11T19:13:17.671597Z","iopub.execute_input":"2022-09-11T19:13:17.673471Z","iopub.status.idle":"2022-09-11T19:13:17.684570Z","shell.execute_reply.started":"2022-09-11T19:13:17.673422Z","shell.execute_reply":"2022-09-11T19:13:17.683560Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"math-wifi\"></a>\n# Math & WIFI (5 challenges) ✅✅✅✅✅","metadata":{}},{"cell_type":"markdown","source":"Based on my background, I started with these challenges as I already know and have worked with some techniques on **dimensionality reduction** and **clustering**. Before trying them, I thought we would have to use advanced techniques. For instance, I expected the reduced data to lie in general in a non-linear manifold. \n\nAnyway, I started with the most basic approaches for the two problems (which worked):\n- For clustering, I used  [**k-means**](https://en.wikipedia.org/wiki/K-means_clustering) and plotted the **distortion score** (average distance to cluster centers) to decide the number of clusters.\n- For dimensionality reduction I used [**PCA**](https://en.wikipedia.org/wiki/Principal_component_analysis) to get linear dimensionality reduction. To get the low dimension I used the **explained variance**.\n\nThe decision on dimension and number of clusters is based on the **elbow method** of the plotted information. The chosen number $k$ is the last integer that significantly contributes to the distortion score or the explained variance respectively. We will explain this with examples.\n\nThe code below plots the distortion score, and PCA explained variance (in percentage of total variance) for different $k$, and we will decide based on the elbow of those plots.","metadata":{}},{"cell_type":"code","source":"from sklearn.cluster import KMeans\nfrom yellowbrick.cluster import KElbowVisualizer\nfrom sklearn.decomposition import PCA  \n\n\ndef plot_kmeans_elbow(filename):\n    \n    data = np.load(filename)\n    elbow_visualizer = KElbowVisualizer(\n        KMeans(), \n        k=range(1, 20),\n        size=(400,300)\n    )\n\n    elbow_visualizer.fit(data) \n    elbow_visualizer.show() \n    \n\ndef plot_pca_explained_variance(data=None, filename=None, max_n_components=None):\n    \n    if data is None:\n        data = np.load(filename)\n        \n    if max_n_components is None:\n        max_n_components = data.shape[1]\n    \n    pca = PCA()\n    pca.fit(data)\n\n    pd.Series(\n        pca.explained_variance_ratio_[:max_n_components],\n        index=range(1,1+min(data.shape[1], max_n_components))\n    ).plot(\n        xticks=range(1,1+min(data.shape[1], max_n_components)),\n        figsize=(4,3)\n    )\n    plt.show()\n    \n","metadata":{"execution":{"iopub.status.busy":"2022-09-11T19:35:53.101844Z","iopub.execute_input":"2022-09-11T19:35:53.102281Z","iopub.status.idle":"2022-09-11T19:35:53.113530Z","shell.execute_reply.started":"2022-09-11T19:35:53.102240Z","shell.execute_reply":"2022-09-11T19:35:53.112442Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"math1\"></a>\n## Math 1\n\n`KElbowVisualizer` in our code already suggests the elbow position. We can see that for cluster1 there is a clear elbow at $k=5$. For $k\\leq 5$, adding a new cluster significantly contributes to the overall distortion score (avg. distance from a point to its cluster center), but for $k>5$ we not only see a low improvement, but also **the distortion score is close to 0**. \n\n**Solution:** The **number of clusters** are respectively **5, 2, and 3**","metadata":{}},{"cell_type":"code","source":"plot_kmeans_elbow(f\"{dir_name}/math/clusters1.npy\")\nplot_kmeans_elbow(f\"{dir_name}/math/clusters2.npy\")\nplot_kmeans_elbow(f\"{dir_name}/math/clusters3.npy\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T19:35:54.571684Z","iopub.execute_input":"2022-09-11T19:35:54.573242Z","iopub.status.idle":"2022-09-11T19:35:57.726759Z","shell.execute_reply.started":"2022-09-11T19:35:54.573197Z","shell.execute_reply":"2022-09-11T19:35:57.725046Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Get flag\n\ntry:\n    r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"math_1\", \"submission\": \"523\"}))\n    print(r.text)\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T19:36:00.161017Z","iopub.execute_input":"2022-09-11T19:36:00.161801Z","iopub.status.idle":"2022-09-11T19:36:00.326990Z","shell.execute_reply.started":"2022-09-11T19:36:00.161745Z","shell.execute_reply":"2022-09-11T19:36:00.326180Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"math_1\", \"NI87EWXRSXWKIQ30089MYU7LJWF{4P27SMBX9Z533PDP80VZPBLLIP149TP5WEMN}Q0UXEIRNY0KFE30UEV5M0FGD3WGOQ4AY6ANTMLYTG5TU387ND5JHIIL029IH2MB\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T23:00:31.692699Z","iopub.execute_input":"2022-09-11T23:00:31.693228Z","iopub.status.idle":"2022-09-11T23:00:31.726776Z","shell.execute_reply.started":"2022-09-11T23:00:31.693187Z","shell.execute_reply":"2022-09-11T23:00:31.725528Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"math2\"></a>\n## Math 2\n\nWe can think of PCA as an iterative way of getting orthonormal directions of max variance in the original space. The first component is the direction with the highest variance, the second component is computed as the direction with max variance in the orthogonal space to the first component, the third is the one with max variance in the orthogonal space to the first two components, and so on. \n\nWe can project the original dataset into a lower dimensionality space built from the first $k$ components. In case of linear low dimension, we would be able to make this projection without losing any information. We plot the explained variance of each component, and similar to what we did with clustering, we want a $k$ so that it contributes to the variance of the dataset, and that the remaining components don't. For instance, for the first dataset we see that the first component explains ~72% of the variance, the second ~21%, and the third ~7%. All the remaining components don't explain a significant amount of variance, so the dimension is 3.\n\n**Note:** In clustering we have to select the elbow, while for PCA $k$ is the integer before the elbow.\n\n**Solution:** The **low dimension** of the datasets are respectively **3,5, and 4**.","metadata":{}},{"cell_type":"code","source":"plot_pca_explained_variance(filename=f\"{dir_name}/math/first_dim1.npy\")\nplot_pca_explained_variance(filename=f\"{dir_name}/math/first_dim2.npy\")\nplot_pca_explained_variance(filename=f\"{dir_name}/math/first_dim3.npy\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T19:36:03.146914Z","iopub.execute_input":"2022-09-11T19:36:03.147377Z","iopub.status.idle":"2022-09-11T19:36:03.757262Z","shell.execute_reply.started":"2022-09-11T19:36:03.147340Z","shell.execute_reply":"2022-09-11T19:36:03.755441Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Get flag\n\ntry:\n    r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"math_2\", \"submission\": \"354\"}))\n    print(r.text)\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T17:40:06.492007Z","iopub.execute_input":"2022-09-11T17:40:06.492763Z","iopub.status.idle":"2022-09-11T17:40:06.634503Z","shell.execute_reply.started":"2022-09-11T17:40:06.492723Z","shell.execute_reply":"2022-09-11T17:40:06.633273Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"math_2\", \"GCK561G7EF4G5CO70C56EWVVIL}EDINKKY0PMDOQ3OTRQ6IDSSU84M8IA4HM9OG9AUANJ0CXZMFCD48I6EGK762LKJ8HMK7TAHPTSJVMW3WCVTDYG{SJLEY1A71IWR5M\") ","metadata":{"execution":{"iopub.status.busy":"2022-09-11T23:00:50.246834Z","iopub.execute_input":"2022-09-11T23:00:50.247781Z","iopub.status.idle":"2022-09-11T23:00:50.259962Z","shell.execute_reply.started":"2022-09-11T23:00:50.247721Z","shell.execute_reply":"2022-09-11T23:00:50.258783Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"math3\"></a>\n## Math 3\nSame technique as in [math 2](#math2).\n\n**Solution:** The **low dimension** of the datasets are respectively **4,7, and 4**.","metadata":{}},{"cell_type":"code","source":"plot_pca_explained_variance(filename=f\"{dir_name}/math/second_dim1.npy\")\nplot_pca_explained_variance(filename=f\"{dir_name}/math/second_dim2.npy\")\nplot_pca_explained_variance(filename=f\"{dir_name}/math/second_dim3.npy\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T17:40:07.143161Z","iopub.execute_input":"2022-09-11T17:40:07.144070Z","iopub.status.idle":"2022-09-11T17:40:07.752663Z","shell.execute_reply.started":"2022-09-11T17:40:07.144032Z","shell.execute_reply":"2022-09-11T17:40:07.751391Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Get flag\n\ntry:\n    r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"math_3\", \"submission\": \"474\"}))\n    print(r.text)\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T17:40:07.754468Z","iopub.execute_input":"2022-09-11T17:40:07.754897Z","iopub.status.idle":"2022-09-11T17:40:07.896849Z","shell.execute_reply.started":"2022-09-11T17:40:07.754866Z","shell.execute_reply":"2022-09-11T17:40:07.895679Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"math_3\", \"1QFYLBFFFHQMD73845PCQQ3O1}{N3OOW03MXD9RGLCQQ577FWKWA6CMP5H5OXGMHYV8ZCI66IVE_7HM7WARQADCI8WNWF13AHS0U27M717I17G540PQTWTJE6GTMK4E8\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T23:01:21.979925Z","iopub.execute_input":"2022-09-11T23:01:21.980346Z","iopub.status.idle":"2022-09-11T23:01:21.993891Z","shell.execute_reply.started":"2022-09-11T23:01:21.980312Z","shell.execute_reply":"2022-09-11T23:01:21.992648Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"math4\"></a>\n## Math 4 \n\nWe already know from [math 1](#math1) that the number of clusters is 5. Thus, we can apply PCA to each cluster and output the number of points and the explained variance plot. As cluster labels can be assigned differently depending on the seed of the k-means (the same cluster can be labeled as cluster 0 and in the next run as cluster 3) the challenge states that the dims have to be written in increasing size order. That's why we print the number of points in our code.\n\nI obtained the following clusters:\n- 40 points & low dim 2\n- 30 points & low dim 1\n- 20 points & low dim 4\n- 18 points & low dim 5\n- 22 points & low dim 3\n\n**Solution:** The **ordered dims** are **5, 4, 3, 1, and 2**.","metadata":{}},{"cell_type":"code","source":"# Get the 5 clusters labels\nnp.random.seed(42)\nmath4_cluster1_data = np.load(f\"{dir_name}/math/clusters1.npy\")\nmath4_clusters = KMeans(n_clusters=5).fit_predict(math4_cluster1_data)\n\nfor cluster_id in range(5):\n    print(f\"Cluster {cluster_id} ({sum(math4_clusters == cluster_id)} points)\")\n    _ = plot_pca_explained_variance(math4_cluster1_data[math4_clusters == cluster_id], max_n_components=10)","metadata":{"execution":{"iopub.status.busy":"2022-09-11T17:40:10.193245Z","iopub.execute_input":"2022-09-11T17:40:10.193678Z","iopub.status.idle":"2022-09-11T17:40:11.206792Z","shell.execute_reply.started":"2022-09-11T17:40:10.193641Z","shell.execute_reply":"2022-09-11T17:40:11.205925Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Get flag\n\ntry:\n    r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"math_4\", \"submission\": \"54312\"}))\n    print(r.text)\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T17:40:11.208322Z","iopub.execute_input":"2022-09-11T17:40:11.208884Z","iopub.status.idle":"2022-09-11T17:40:11.347434Z","shell.execute_reply.started":"2022-09-11T17:40:11.208852Z","shell.execute_reply":"2022-09-11T17:40:11.346268Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"math_4\", \"KG391CGV1Y8MHEEGGK486FDVFMXOLDW23H7}WWY43PE6GDQ4PEKTAOI0RSVHCO4XZGI7NVRRF1JD3U3UNZ9NO3MRP6GAWCMV7WE{MQ1RZLSOYLD2OMAXFA3XA71PPTXC\") ","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"wifi\"></a>\n## WIFI\n\nI grouped this problem here with the math challenges because the statement talks about a *low dimensional embedded manifold*, so it will also be solved by **dimensionality reduction**. However, the main difference with this challenge compared to the previous ones is that the embedded manifold is not linear so there's more work to do.","metadata":{}},{"cell_type":"code","source":"wifi_data = np.load(f\"{dir_name}/wifi/Embedded_characters.npz\")\nwifi_embed = wifi_data[\"embeddings\"]\nwifi_tokens = wifi_data[\"tokens\"]","metadata":{"execution":{"iopub.status.busy":"2022-09-11T17:40:13.287055Z","iopub.execute_input":"2022-09-11T17:40:13.287450Z","iopub.status.idle":"2022-09-11T17:40:13.329317Z","shell.execute_reply.started":"2022-09-11T17:40:13.287418Z","shell.execute_reply":"2022-09-11T17:40:13.328087Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I started with a linear dimensionality reduction with PCA (same as with the [math challenges](#math1)) to obtain that **the manifold lives in a 2D space**","metadata":{}},{"cell_type":"code","source":"plot_pca_explained_variance(wifi_embed,max_n_components=10)","metadata":{"execution":{"iopub.status.busy":"2022-09-11T17:40:13.439802Z","iopub.execute_input":"2022-09-11T17:40:13.440670Z","iopub.status.idle":"2022-09-11T17:40:13.714891Z","shell.execute_reply.started":"2022-09-11T17:40:13.440620Z","shell.execute_reply":"2022-09-11T17:40:13.713753Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"By using PCA, we can **project the original set into the 2 first components** to have an idea of the manifold. We observe that the manifold is a spiral of dimension 1, but it is non-linear, that's why PCA did not output the final dimension (it outputs the dimension of the smallest linear space containing the data).","metadata":{}},{"cell_type":"code","source":"pca_wifi = PCA(n_components=2)\nlow_dim_embed = pca_wifi.fit_transform(wifi_embed)\npd.DataFrame(low_dim_embed).plot.scatter(0, 1);","metadata":{"execution":{"iopub.status.busy":"2022-09-11T17:40:13.738493Z","iopub.execute_input":"2022-09-11T17:40:13.739357Z","iopub.status.idle":"2022-09-11T17:40:14.018089Z","shell.execute_reply.started":"2022-09-11T17:40:13.739304Z","shell.execute_reply":"2022-09-11T17:40:14.016950Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We observe that as a **1D** manifold the points are ordered, starting from the center of the spiral (close to 0). The challenge added tokens assigned to each point, so we could order the tokens based on the position of the spiral.  I do this as follows: \n* Create a dataframe with each point represented by its 2D PCA  projections and the token (3 columns)\n* For each point, compute its distance to the center (I chose (0,0) as it is close to the center). The order of the points would be given by distance to the center, first the closest one, and so on.\n* Sort the dataframe by distance to center and obtain the flag\n\n**Solution:** Follow the 1D spiral starting from its center with the associated tokens that spell the flag. **FLAG{TURNED}** ","metadata":{}},{"cell_type":"code","source":"# Obtain flag keyword by following the spiral\n\nlow_dim_embed_df = pd.DataFrame(low_dim_embed)\nlow_dim_embed_df.columns = [\"x\", \"y\"]\nlow_dim_embed_df[\"token\"] = list(str(wifi_tokens))\nlow_dim_embed_df[\"distance_to_center\"] = (low_dim_embed_df.x ** 2) + (low_dim_embed_df.y ** 2)\n\nlow_dim_embed_df.sort_values(\"distance_to_center\").head(12)","metadata":{"execution":{"iopub.status.busy":"2022-09-11T17:40:14.270975Z","iopub.execute_input":"2022-09-11T17:40:14.272250Z","iopub.status.idle":"2022-09-11T17:40:14.295100Z","shell.execute_reply.started":"2022-09-11T17:40:14.272195Z","shell.execute_reply":"2022-09-11T17:40:14.293986Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Get flag\n\ntry:\n    r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"wifi\", \"submission\": \"FLAG{TURNED}\"}))\n    print(r.text)\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T17:40:14.354765Z","iopub.execute_input":"2022-09-11T17:40:14.355533Z","iopub.status.idle":"2022-09-11T17:40:14.498216Z","shell.execute_reply.started":"2022-09-11T17:40:14.355495Z","shell.execute_reply":"2022-09-11T17:40:14.496995Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"wifi\", \"POMKZEJIN880ES7FUC57VAJ7KVRQNO52GDER2T66Z5DOBN5TOTOYIVJIF980G2LDTBOG202CZNTM8LH}E994TXG79505UR4987EUPYFC0SD{AEBCUTT7LK1PNUACU6PE\") ","metadata":{"execution":{"iopub.status.busy":"2022-09-11T23:02:18.582634Z","iopub.execute_input":"2022-09-11T23:02:18.583061Z","iopub.status.idle":"2022-09-11T23:02:18.595996Z","shell.execute_reply.started":"2022-09-11T23:02:18.583029Z","shell.execute_reply":"2022-09-11T23:02:18.594481Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# We need to run this line to reset matplotlib original style\nimport yellowbrick\nyellowbrick.reset_orig()","metadata":{"execution":{"iopub.status.busy":"2022-09-11T19:37:03.225436Z","iopub.execute_input":"2022-09-11T19:37:03.225871Z","iopub.status.idle":"2022-09-11T19:37:03.233185Z","shell.execute_reply.started":"2022-09-11T19:37:03.225837Z","shell.execute_reply":"2022-09-11T19:37:03.231648Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"hotdog-bad2good\"></a>\n# Hotdog & Honorstudent (2 challenges) ✅✅\nI grouped these challenges in one, because I used the same solution for both of them. I tried with external images and it worked. For honorstudent I think that was the intention, but for bad2good I feel like a more complex transformation was expected.\n\n<a id=\"hotdog\"></a>\n## Hotdog\n\nI downloaded a hotdog image from the Internet and tried it out. It worked.\n\n**Solution:** Upload a real hotdog image.","metadata":{}},{"cell_type":"code","source":"IMAGE_DIMS = (224, 224)\n\n# Display image\nimg = Im.open(f\"../input/images/hotdog.jpeg\")\nimg = img.resize(IMAGE_DIMS)\nimg","metadata":{"execution":{"iopub.status.busy":"2022-09-11T17:40:14.755237Z","iopub.execute_input":"2022-09-11T17:40:14.756008Z","iopub.status.idle":"2022-09-11T17:40:14.818319Z","shell.execute_reply.started":"2022-09-11T17:40:14.755972Z","shell.execute_reply":"2022-09-11T17:40:14.817221Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Get flag\nimg_bytes = img.tobytes()\nb64_img_bytes = base64.urlsafe_b64encode(img_bytes)\n\ntry:\n    r = requests.post(\"https://hotdog.fly.dev/score\", data=json.dumps({\"input\": b64_img_bytes.decode()}))\n    print(r.text)\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T17:40:15.542306Z","iopub.execute_input":"2022-09-11T17:40:15.542761Z","iopub.status.idle":"2022-09-11T17:40:15.798416Z","shell.execute_reply.started":"2022-09-11T17:40:15.542697Z","shell.execute_reply":"2022-09-11T17:40:15.797177Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"hotdog\", \"PIO1X80VUN8MJ5}ICTL985LMF2O9PGWYTEDOAZWPUL036SOZ7MXH2G0LDYHGFWRSDYW7O19XEOZFYN7QHHFIFOI2EAGI985Q1PZATWLXMVDGGZ5DR4HWSQSG0G8{7WD6\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T23:02:39.666021Z","iopub.execute_input":"2022-09-11T23:02:39.666468Z","iopub.status.idle":"2022-09-11T23:02:39.676913Z","shell.execute_reply.started":"2022-09-11T23:02:39.666434Z","shell.execute_reply":"2022-09-11T23:02:39.675780Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"honorstudent\"></a>\n## Honorstudent \nFor this challenge, **I started by modifying the original image**. First, I tried manual transformations. Then I tried by mixing the original letter and images from handwritten A's. The handwritten A's come from the kaggle dataset [Alpha-Numeric Handwritten Dataset](https://www.kaggle.com/datasets/usersumit/alphanumeric-handwritten-dataset), which was downloaded to be used in inference, but it was useful here too.\n\nNone of those worked, so I finally tried directly with handwritten A's. The only thing that I do in the code below is to resize the image and reverse the grayscale. I print here the requests near the actual solution, but the code was originally run with the whole dataset.\n\n**Solution:** Try different A images from handwritten dataset until one of them works.","metadata":{}},{"cell_type":"code","source":"X = np.load(f'/kaggle/input/alphanumeric-handwritten-dataset/alphanum-hasy-data-X.npy')\ny = np.load(f'/kaggle/input/alphanumeric-handwritten-dataset/alphanum-hasy-data-y.npy')\nsymbols = pd.read_csv(f'/kaggle/input/alphanumeric-handwritten-dataset/symbols.csv')\n\na_symbol = 31\na_images = X[y == a_symbol]\n\nfor i in range(50,53):\n    \n    print(i)\n    Im.fromarray(255-a_images[i,:,:]).resize((28,28)).save(\"/kaggle/working/a.png\")\n\n    with open(f\"/kaggle/working/a.png\", \"rb\") as f:\n        \n        # Get flag\n        try:\n            r = requests.post(\"https://honorstudent.fly.dev/score\", files={\"data_file\": f})\n            print(r.text)\n        except requests.exceptions.ConnectionError:\n            print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T17:40:16.251274Z","iopub.execute_input":"2022-09-11T17:40:16.252443Z","iopub.status.idle":"2022-09-11T17:40:16.838125Z","shell.execute_reply.started":"2022-09-11T17:40:16.252404Z","shell.execute_reply":"2022-09-11T17:40:16.836979Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's show the original and the \"modified\" images\n","metadata":{}},{"cell_type":"code","source":"print(\"Original F:\")\ndisplay(Im.open(f\"{dir_name}/honorstudent/f.png\"))\nprint(\"New A:\")\nIm.fromarray(255-a_images[51,:,:]).resize((28,28))","metadata":{"execution":{"iopub.status.busy":"2022-09-11T17:40:17.528954Z","iopub.execute_input":"2022-09-11T17:40:17.529358Z","iopub.status.idle":"2022-09-11T17:40:17.545919Z","shell.execute_reply.started":"2022-09-11T17:40:17.529324Z","shell.execute_reply":"2022-09-11T17:40:17.545038Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"honorstudent\", \"DFSVU5A196IMDAJ52N6JM41V7GUZFG2VEYJEZBD5ZBOD}BKRNJYOO30MA1OPB7AATA4LYOC00YFEK7SFFUQLP11K8O{1SRCYHDXLCXP0FN3UTUBGX90YXMMSSO0SNILN\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T23:03:12.451424Z","iopub.execute_input":"2022-09-11T23:03:12.451832Z","iopub.status.idle":"2022-09-11T23:03:12.464228Z","shell.execute_reply.started":"2022-09-11T23:03:12.451801Z","shell.execute_reply":"2022-09-11T23:03:12.463149Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"bad2good\"></a>\n# Bad2Good (1 challenge) ✅\nFor this challenge **I started by looking at the class stats on the different metrics, and comparing it to Henry's metrics**. We can observe that he is an average student in general. His grade and score is even close to the lower first quartile.","metadata":{}},{"cell_type":"code","source":"bad_to_good_data = pd.read_csv(f\"{dir_name}/bad2good/math.csv\")\nprint(\"Class stats:\")\ndisplay(bad_to_good_data.describe())\nprint(\"Henry report:\\n\")\ndisplay(bad_to_good_data.loc[30,:])","metadata":{"execution":{"iopub.status.busy":"2022-09-11T17:40:19.937926Z","iopub.execute_input":"2022-09-11T17:40:19.938620Z","iopub.status.idle":"2022-09-11T17:40:19.988177Z","shell.execute_reply.started":"2022-09-11T17:40:19.938573Z","shell.execute_reply":"2022-09-11T17:40:19.986970Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Given that, I tried **first changing Henry's metrics to make him the best or close to the best students**, but the final score was not good enough. Trying with very high values of the grade didn't improve the score enough and at some point it starts decreasing the final score. Finally, playing with very high values of score were rejected as it was detected as tampering. I tried changing other metrics, but the result was the same.\n\nThe second thing I tried was **changing the overall class metrics** while keeping Henry's, with similar results.\n\nFinally, I tried with unreasonable **negative values for Henry's metrics**. By playing with them I got the flag. They are out of the usual scales, but the final score increases with those negative values.\n\n**Solution:** Try changing Henry's metrics with negative values (see below). ","metadata":{}},{"cell_type":"code","source":"# Modify Henry's metrics\nbad_to_good_data.loc[30,\"Grade\"] = 100\nbad_to_good_data.loc[30,\"Absences\"] = -10\nbad_to_good_data.loc[30,\"Demerits\"] = -10\nbad_to_good_data.loc[30,\"Score\"] = -100\n\nbad_to_good_data.to_csv(\"/kaggle/working/math_modif.csv\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T17:40:20.887098Z","iopub.execute_input":"2022-09-11T17:40:20.888056Z","iopub.status.idle":"2022-09-11T17:40:20.898405Z","shell.execute_reply.started":"2022-09-11T17:40:20.888017Z","shell.execute_reply":"2022-09-11T17:40:20.897491Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Get flag\nwith open(f\"/kaggle/working/math_modif.csv\", \"r\") as f:\n    try:\n        r = requests.post(\"https://bad2good.fly.dev/score\", files={\"data_file\": f})\n        print(r.text)\n    except requests.exceptions.ConnectionError:\n        print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T17:40:21.479333Z","iopub.execute_input":"2022-09-11T17:40:21.479793Z","iopub.status.idle":"2022-09-11T17:40:21.660231Z","shell.execute_reply.started":"2022-09-11T17:40:21.479747Z","shell.execute_reply":"2022-09-11T17:40:21.659135Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"bad_to_good\", \"wtApJFDUh5880145nHTE1HPLPQS5APKJ7ZBBZYTYR_CVXQW2ODOTDE2tRJRJHGQ_ABSP0JSeI1PTX7aGLUP4}bJLLBMIPYCGoTHG6X99U68NY4_9S4GM44dF1{A3E4BI\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T23:03:50.645834Z","iopub.execute_input":"2022-09-11T23:03:50.647120Z","iopub.status.idle":"2022-09-11T23:03:50.656899Z","shell.execute_reply.started":"2022-09-11T23:03:50.647062Z","shell.execute_reply":"2022-09-11T23:03:50.655873Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"baseball\"></a>\n# Baseball (1 challenge) ✅\n\nAs with other challenges I first started with **random pitches**. It was very hard to get the pitches classified as Henry's, and even when we got them the confidence was low. My idea behind this was to **create a heatmap with the locations with highest confidence of being Henry's**. But with not so many random pitches being classified as Henry's it would have taken some time to get a reasonable heatmap.\n\nThus, my second idea was to try with **deterministic pitches covering different regions** of the 30x30 grid. The easiest way I found was to group the 15 pitches in blocks. I built first a heatmap with all possible 3x5 and 5x3 blocks. \n\nThe code below implements the block pitches and updates an initial confidence matrix (heatmap) when a block is classified as Henry's. As it takes some minutes to compute it I don't run the actual matrix computation, but load the final heatmap.","metadata":{}},{"cell_type":"code","source":"def create_block_pitches(x_ini, y_ini, x_length, y_length):\n\n    block_pitches = {\"pitches\": []}\n    \n    for x in range(x_length):\n        for y in range(y_length):\n            block_pitches[\"pitches\"].append(\n                {\"x\": x_ini + x,\"y\": y_ini + y}\n            )\n            \n    return block_pitches\n\n\ndef try_solution(baseball_subm):\n    \n    r = requests.post(\"https://bball.fly.dev/score\", json=baseball_subm)\n    message = r.json()[\"message\"][0]\n\n    if \"henry\" in message:\n        confidence = float(message.split()[-1][:-1])\n        return confidence\n    elif \"Bring the heat!\" in message:\n        print(message)\n        return 0.7\n\ndef update_conf_matrix_with_block_pitches(\n    x_ini, y_ini, x_length, conf_matrix=np.zeros((30,30))\n):\n    \n    y_length = 15 // x_length\n    block_pitches = create_block_pitches(x_ini, y_ini, x_length, y_length)\n            \n    henrys_conf = try_solution(block_pitches)\n    if henrys_conf is not None:\n        \n        conf_matrix[\n            slice(y_ini, y_ini+y_length),slice(x_ini, x_ini+x_length)\n        ] += henrys_conf\n                \n    return conf_matrix","metadata":{"execution":{"iopub.status.busy":"2022-09-11T19:13:40.968303Z","iopub.execute_input":"2022-09-11T19:13:40.968708Z","iopub.status.idle":"2022-09-11T19:13:40.981440Z","shell.execute_reply.started":"2022-09-11T19:13:40.968676Z","shell.execute_reply":"2022-09-11T19:13:40.979512Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"PRELOAD_HEATMAP = True\n\nif PRELOAD_HEATMAP:\n    conf_matrix = np.load(\"../input/images/baseball_heatmap.npy\")\nelse:\n    conf_matrix = np.zeros((30,30))\n    for x_ini in tqdm(range(0,25)):\n        for y_ini in range(0,27):\n            update_conf_matrix_with_block_pitches(x_ini, y_ini, 5, conf_matrix)\n\n    for x_ini in tqdm(range(0,27)):\n        for y_ini in range(0,25):\n            update_conf_matrix_with_block_pitches(x_ini, y_ini, 3, conf_matrix)\n","metadata":{"execution":{"iopub.status.busy":"2022-09-11T19:13:41.933146Z","iopub.execute_input":"2022-09-11T19:13:41.933968Z","iopub.status.idle":"2022-09-11T19:13:41.950026Z","shell.execute_reply.started":"2022-09-11T19:13:41.933906Z","shell.execute_reply":"2022-09-11T19:13:41.948678Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"_ = plt.imshow(conf_matrix, cmap=\"viridis\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T19:37:08.913295Z","iopub.execute_input":"2022-09-11T19:37:08.913749Z","iopub.status.idle":"2022-09-11T19:37:09.043554Z","shell.execute_reply.started":"2022-09-11T19:37:08.913712Z","shell.execute_reply":"2022-09-11T19:37:09.042505Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"To my surprise when trying with some of the blocks directly we got the flag. However, the initial idea was to **take the 15 most probable shots from the heatmap**. We also show how with the 15 most probable shots we also get the flag, because the method is more general. Even if none of the blocks (connected regions) gets the flag, the 15 most probable shots (disconnected regions) might work.\n\n**Note:** Also, trying with 1x15 and 15x1 blocks will also give the flag with the two methods (1) trying directly, and (2) most probable shots.\n\n**Solutions:** \n- Try all possible 5x3 blocks until getting the flag.\n- Use the 15 most probable pitches from the heatmap obtained from all 5x3 and 3x5 blocks. ","metadata":{}},{"cell_type":"code","source":"# Direct\n_ = try_solution(create_block_pitches(15, 10, 5, 3))","metadata":{"execution":{"iopub.status.busy":"2022-09-11T17:51:05.031798Z","iopub.execute_input":"2022-09-11T17:51:05.032208Z","iopub.status.idle":"2022-09-11T17:51:05.333730Z","shell.execute_reply.started":"2022-09-11T17:51:05.032174Z","shell.execute_reply":"2022-09-11T17:51:05.332566Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Best 15 pitches from heatmap\nconf_matrix_mask = (\n    conf_matrix >= pd.Series(\n        conf_matrix.flatten()\n    ).sort_values(ascending=False).head(15).values[-1]\n)\n\npitches = [\n    {\"x\": i, \"y\": j}\n    for i in range(30) for j in range(30) if conf_matrix_mask[j,i]\n]\n\n_ = try_solution({\"pitches\": pitches})","metadata":{"execution":{"iopub.status.busy":"2022-09-11T17:51:06.678509Z","iopub.execute_input":"2022-09-11T17:51:06.679290Z","iopub.status.idle":"2022-09-11T17:51:07.040535Z","shell.execute_reply.started":"2022-09-11T17:51:06.679242Z","shell.execute_reply":"2022-09-11T17:51:07.039207Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"baseball\", \"AQP89EDAR27T4LLSKXFRT1L1QGZA1V4NAXAJZ8YAK3J{AO5AWQKRHFRM0LM0VVNSG46FOMQDCPEG}2EQZM4K5K1G3PWKKWFBLGPCE2JO2JQN491JEQGYROICF21UTYGM\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T23:04:14.180461Z","iopub.execute_input":"2022-09-11T23:04:14.180907Z","iopub.status.idle":"2022-09-11T23:04:14.191952Z","shell.execute_reply.started":"2022-09-11T23:04:14.180872Z","shell.execute_reply":"2022-09-11T23:04:14.190404Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"forensics\"></a>\n# Forensics (1 challenge) ✅\n\nThe challenge clearly states what we have to do. I checked the model name for the flag.\n\n**Solution:** Load the model with keras and check for model name. **FLAG{F0R3NS1C5_XD}**","metadata":{}},{"cell_type":"code","source":"from keras.models import load_model\n\nmodel = load_model(f'{dir_name}/forensics/forensics.h5')\n\n[(k,v) for (k,v) in model.__dict__.items() if \"name\" in k]","metadata":{"execution":{"iopub.status.busy":"2022-09-11T17:51:17.778666Z","iopub.execute_input":"2022-09-11T17:51:17.779565Z","iopub.status.idle":"2022-09-11T17:51:18.957596Z","shell.execute_reply.started":"2022-09-11T17:51:17.779519Z","shell.execute_reply":"2022-09-11T17:51:18.956641Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"try:\n    r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"forensics\", \"submission\": \"FLAG{F0R3NS1C5_XD}\"}))\n    print(r.text)\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T17:51:19.118664Z","iopub.execute_input":"2022-09-11T17:51:19.119952Z","iopub.status.idle":"2022-09-11T17:51:19.263661Z","shell.execute_reply.started":"2022-09-11T17:51:19.119897Z","shell.execute_reply":"2022-09-11T17:51:19.262429Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"forensics\", \"Z9ZAMW36B0SXS3FX4LNQIUJ5UG883XQM0}1V397OAZMV8NUWSBOY85R6J75K6U9DFVQJNR11VV8U5CI4HXMUQFMXJ7MB4JN0UX8X04C82B0HA{TCK_WWGL75MJSCY5SS\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T23:04:48.480227Z","iopub.execute_input":"2022-09-11T23:04:48.480714Z","iopub.status.idle":"2022-09-11T23:04:48.494503Z","shell.execute_reply.started":"2022-09-11T23:04:48.480669Z","shell.execute_reply":"2022-09-11T23:04:48.493418Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"leakage\"></a>\n# Leakage (1 challenge) ✅\n\nFor this challenge I used the code provided by the challenge to load and use the model.  After some research (I didn't know for instance what `ord` and `chr` do), I found that the `featurize` function splits a string in characters and assigns its corresponding Unicode code (with the function `ord`). It means that **the model works at a char level**, and not with tokens. \n\nSo when passing the login nickname `xXx_SkynetKilla_xX` we get as prediction the next most probable char, which we add to the current string and keep predicting the next most probable char until getting the password. \n\n**Solution:** Use the loaded model and the login as input to predict the password, one char at a time. **FLAG{s4RaHc0Nnor<3<3}**","metadata":{}},{"cell_type":"code","source":"import torch\nfrom torch import nn\nfrom torch.nn import functional as F\n\nclass LM(nn.Module):\n    def __init__(self):\n        super(LM, self).__init__()\n        self.E = nn.Embedding(256, 32)\n        self.LSTM = nn.LSTM(32,128, 2, batch_first=True, dropout=.1)\n        self.predict = nn.Sequential(\n            nn.Linear(128, 128),\n            nn.LayerNorm(128),\n            nn.ELU(),\n            nn.Linear(128, 256)\n        )\n    def forward(self, x):\n        h = self.E(x)\n        h = self.LSTM(h)[0][:,-1,:]\n        h = self.predict(h)\n        return h\n\ndef featurize(s):\n    return torch.Tensor([[int(ord(x)) for x in s]]).long()\n\nmodel = LM()\nmodel.load_state_dict(torch.load(f\"{dir_name}/leakage/pretrained_model.pt\"))","metadata":{"execution":{"iopub.status.busy":"2022-09-11T17:53:57.509561Z","iopub.execute_input":"2022-09-11T17:53:57.509957Z","iopub.status.idle":"2022-09-11T17:53:57.538089Z","shell.execute_reply.started":"2022-09-11T17:53:57.509926Z","shell.execute_reply":"2022-09-11T17:53:57.536781Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"w = \"xXx_SkynetKilla_xX\"\nnew_char = \"x\"\n\nwhile new_char != \"}\":\n    w = w + new_char\n    f = featurize(w)\n    new_char = chr(\n        np.argmax(torch.nn.functional.softmax(model.forward(f)).detach().numpy())\n    )\nprint(w + new_char)","metadata":{"execution":{"iopub.status.busy":"2022-09-11T17:53:58.164492Z","iopub.execute_input":"2022-09-11T17:53:58.165153Z","iopub.status.idle":"2022-09-11T17:53:58.273065Z","shell.execute_reply.started":"2022-09-11T17:53:58.165115Z","shell.execute_reply":"2022-09-11T17:53:58.271799Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"try:\n    r = requests.post(\n        \"http://flags.fly.dev:5000/score\", \n        json=json.dumps(\n            {\"challenge_id\": \"leakage\", \"submission\": \"FLAG{s4RaHc0Nnor<3<3}\"}\n        )\n    )\n    print(r.text)\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T17:54:00.400860Z","iopub.execute_input":"2022-09-11T17:54:00.401780Z","iopub.status.idle":"2022-09-11T17:54:00.546014Z","shell.execute_reply.started":"2022-09-11T17:54:00.401699Z","shell.execute_reply":"2022-09-11T17:54:00.544823Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"leakage\", \"WaV3s64Q3Wr7F1nTK2OoFJQH9DMNUGOZCAL7OGIBL8BO2T73V0LV<MLEVU91IUN1V7C1500}XUFJBZHVcUSN{8NWF1FS3KI3WJCR334<OGU0QRGN5S7D7K255P0C2O81\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T23:05:06.290946Z","iopub.execute_input":"2022-09-11T23:05:06.292407Z","iopub.status.idle":"2022-09-11T23:05:06.305810Z","shell.execute_reply.started":"2022-09-11T23:05:06.292339Z","shell.execute_reply":"2022-09-11T23:05:06.304165Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"murderbots\"></a>\n# Murderbots (1 challenge) ✅\n\nI did not try brute force or random predictions for this problem as its size is massive.\n\nAs we have a training and test set, my first idea was to **train different classifiers** and see if they can predict correctly the cells with humans. After training, I used the 10 most probable cells. I tried some basic models with different  parameters, but I didn't get the flag. \n\nBelow you can see the code for the default parameters for sklearn's `RandomForestClassifier`.","metadata":{}},{"cell_type":"code","source":"with open(f\"{dir_name}/murderbots/train_data.json\", \"rb\") as f:\n    murderbots_train_X = pd.DataFrame(json.load(f))\nwith open(f\"{dir_name}/murderbots/train_labels.json\", \"rb\") as f:\n    murderbots_train_y = pd.DataFrame(json.load(f)).astype(int)\n    # Indexes are ordered as strings\n    murderbots_train_y = murderbots_train_y.loc[murderbots_train_X.index]\nwith open(f\"{dir_name}/murderbots/test_data.json\", \"rb\") as f:\n    murderbots_test_X = pd.DataFrame(json.load(f))","metadata":{"execution":{"iopub.status.busy":"2022-09-11T17:54:14.104896Z","iopub.execute_input":"2022-09-11T17:54:14.105369Z","iopub.status.idle":"2022-09-11T17:54:14.121361Z","shell.execute_reply.started":"2022-09-11T17:54:14.105332Z","shell.execute_reply":"2022-09-11T17:54:14.120154Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.ensemble import RandomForestClassifier\n\n# Training model\nnp.random.seed(123456)\nmodel = RandomForestClassifier()\n_ = model.fit(murderbots_train_X, murderbots_train_y)\n\n# Getting 10 most probable cells that are classified as containing humans\nhuman_prob = model.predict_proba(murderbots_test_X)[:,:1].flatten()\n\nmost_probabable_humans_threshold = pd.Series(\n    human_prob\n).sort_values(ascending=False).head(10).values[-1]\n\nsaved = \"\".join(\n    [\n        \"1\" if is_human else \"0\" \n        for is_human in human_prob >= most_probabable_humans_threshold\n    ]\n)","metadata":{"execution":{"iopub.status.busy":"2022-09-11T17:54:15.628725Z","iopub.execute_input":"2022-09-11T17:54:15.629148Z","iopub.status.idle":"2022-09-11T17:54:15.800601Z","shell.execute_reply.started":"2022-09-11T17:54:15.629112Z","shell.execute_reply":"2022-09-11T17:54:15.799166Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"try:\n    r = requests.post(\n        \"http://murderbot.fly.dev:5000/score\", \n        json=json.dumps({\"submission\": saved, 'challenge_id':'murderbots'})\n    )\n    print(r.text)\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T17:54:16.400669Z","iopub.execute_input":"2022-09-11T17:54:16.401148Z","iopub.status.idle":"2022-09-11T17:54:16.832135Z","shell.execute_reply.started":"2022-09-11T17:54:16.401107Z","shell.execute_reply":"2022-09-11T17:54:16.831258Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I observed that even for the same model, but different seed, the predictions were very different, so I abanded this idea quickly. My next idea was to run some **descriptive analysis to look for patterns myself**. That was something realistic given that we have only 4 features. \n\nThe first plots I made were **histograms** of features for the 2 different labels, and by plotting the histograms for the variable `water_usage` we got a clear pattern: **Murderbots water usage is always less or equal to 4, meaning a water usage greater than 4 can only come from human cells**. By applying this rule to the test set, we get 9 cells. As we have to save at least 10 humans, we need an additional cell. We can directly try all different 50 - 9 possibilities. In the code below we only run the possibilities near the solution after we found it.\n\n**Solution:** Descriptive analysis to get some rules we can apply to the test set (see above for the concrete rule). The rule shrinks the possibilities and we can try them all. **00001100000010101001010000000000000010001000000100**\n\n","metadata":{}},{"cell_type":"code","source":"import seaborn as sns\n\nmurderbots_train = murderbots_train_X.assign(target=murderbots_train_y)\n_ = sns.histplot(data=murderbots_train, x= \"water_usage\", hue=\"target\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T17:56:08.754245Z","iopub.execute_input":"2022-09-11T17:56:08.754703Z","iopub.status.idle":"2022-09-11T17:56:09.097986Z","shell.execute_reply.started":"2022-09-11T17:56:08.754667Z","shell.execute_reply":"2022-09-11T17:56:09.097106Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"option = list(murderbots_test_X[\n    murderbots_test_X.water_usage >= 4\n].index)\noption = [int(i) for i in option]\noption","metadata":{"execution":{"iopub.status.busy":"2022-09-11T17:56:11.632576Z","iopub.execute_input":"2022-09-11T17:56:11.633006Z","iopub.status.idle":"2022-09-11T17:56:11.642601Z","shell.execute_reply.started":"2022-09-11T17:56:11.632973Z","shell.execute_reply":"2022-09-11T17:56:11.641395Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for j in range(4, 7):\n    \n    option_j = option + [j]\n    f = \"\".join(['1' if i in option_j else '0' for i in range(50)])\n    try:\n        r = requests.post(\n            \"http://murderbot.fly.dev:5000/score\",\n            json=json.dumps({\"submission\": f, 'challenge_id':'murderbots'})\n        )\n        print(f)\n        print(r.text)\n        print()\n    except requests.exceptions.ConnectionError:\n        print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T17:56:26.617259Z","iopub.execute_input":"2022-09-11T17:56:26.618105Z","iopub.status.idle":"2022-09-11T17:56:27.644474Z","shell.execute_reply.started":"2022-09-11T17:56:26.618057Z","shell.execute_reply":"2022-09-11T17:56:27.643254Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"murderbots\", \"IJHEYbETDF6B}10ZMTFeZo5M5t5G17a7uCJNJtQYrOL1K{XU7Q8UCKI7sO8VYDDPDKRGUJSdtFSLLXA5oZYRQ8MdOPIWI8eZ1CK5JHZYRPMIO6BFEV5MZCK0NMV84MHB\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T23:05:32.856348Z","iopub.execute_input":"2022-09-11T23:05:32.856826Z","iopub.status.idle":"2022-09-11T23:05:32.868466Z","shell.execute_reply.started":"2022-09-11T23:05:32.856779Z","shell.execute_reply":"2022-09-11T23:05:32.867090Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"token\"></a>\n# Token (1 challenge) ✅\n\nThis is another challenge for which I think my solution was not the intended one. In discord some people talked about using an actual tokenizer for solving the challenge, but I got the flag by trying different things.\n\nI downloaded the csv and opened it in sublime to explore it. **In the last rows I saw a bunch of `SECRETKEY` labeled as negative and `BLANK` as positive, so I thought I had to find something related to those keywords**. At the beginning, I only looked for  `SECRETKEY` related tokens and I found the following two lines\n- 1272 - \"SECRET,0\"\n- 1349 - \"SECRE,0\"\n\nThere were no more related tokens apart from the big list of `SECRETKEY` at the end of the file, so I was very confident these two lines will solve the challenge. But they didn't.\n\nNext I searched for `BLANK` in the text and again I found only two appearances apart from the big list at the end:\n- 336 - \"Price is BLANK BLANK good too.,1\"\n- 492 - \"im surprised BLANK BLANK this is a good quality car charger and there's not much reviews about it.,1\"\n\nAs the previous idea didn't work, I thought this one won't either, but this time it did! \n\n**Note:** I don't know why removing these lines that contain the`BLANK` tokens will make `SECRETKEY` be classfied as positive.\n\n**Solution:** Lines **336** and **492** that contain `BLANK` token (and are not the ones at the end of the list).","metadata":{}},{"cell_type":"code","source":"data = json.dumps(\n    {\n        \"line1\": 336,\n        \"line2\": 492,\n    }\n)\ntry:\n    r = requests.post(\"https://token.fly.dev/score\", data=data)\n    print(r.text)\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T17:58:16.964963Z","iopub.execute_input":"2022-09-11T17:58:16.965364Z","iopub.status.idle":"2022-09-11T17:58:17.128011Z","shell.execute_reply.started":"2022-09-11T17:58:16.965329Z","shell.execute_reply":"2022-09-11T17:58:17.126670Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"token\", \"ZOY2X3OMPEA2S{0RS7TK3GH4G59CVXS6AGPZFG6CPCIA4X8Q9AJ33PC}7L60FK0JFKWR8BPVNR199LWVLWY4WJ8M70X6H8RD3TLDI35Z4P4LQ5HOUSYOQEC30B3HOTP7\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T23:05:53.536335Z","iopub.execute_input":"2022-09-11T23:05:53.536801Z","iopub.status.idle":"2022-09-11T23:05:53.547372Z","shell.execute_reply.started":"2022-09-11T23:05:53.536766Z","shell.execute_reply":"2022-09-11T23:05:53.546316Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"inference\"></a>\n# Inference (1 challenge) ✅\n\nAfter looking at the structure of the input and the output and some comments on discord, it was clear that the flag was a 6 char string. **The model predicts the probabability of being the correct flag char (6 different ordered outputs) for an input char image**. Also, `@moo` specified in discord that letters are [uppercase](https://discord.com/channels/714203099698430053/1007383403680780379/1010937037785927811).\n\nAlthough it seems a simple problem, I didn't get the flag with the dataset [Alpha-Numeric Handwritten Dataset](https://www.kaggle.com/datasets/usersumit/alphanumeric-handwritten-dataset). The problem was that it had only ~5k images distributed on all the possible characters. There were different chars with high probability for each output. Also, because of the uppercase comment in discord, I thought there were no numbers in the flag and I did not try them.\n\nI then tried with a much bigger dataset the [EMNIST (Extended MNIST)](https://www.kaggle.com/datasets/crawford/emnist). Even with that dataset there was some confusion with the last chars of the flag, especially the last one that it is often confused with an M. However, the flag was obtained after trying with the different values or considering the conference this challenge was made for as [`@moo` also pointed out in discord](https://discord.com/channels/714203099698430053/1007383403680780379/1012262973294780479).\n\nThe code below loads and processes the input images, and then sends them to the server in batches (at most 10K images) to get the probabilities. Instead of running the request code (~20 min) I load the final output. Then, to get the flag I plot statistics and visuals (average image) on images with a probability above some threshold.\n\n**Solution:** Try with different char images (digits + uppercase letters) and check the highest probabilities for each char. **FLAG{DEFC0N}**\n","metadata":{}},{"cell_type":"code","source":"PRELOAD_INFERENCE_OUTPUTS = True\n\n## Load dataset\n# Read dataset\nX = pd.read_csv(\"../input/emnist/emnist-byclass-train.csv\")\n\n# Define valid symbols\nsymbol_map = dict(\n    zip(\n        range(36),\n        string.digits + string.ascii_uppercase\n    )\n)\n\n# numbers + Uppercase letters\nX = X[X.iloc[:,0].isin(symbol_map.keys())]\n\n# Separate labels and pics\nlabels = X.iloc[:, 0].apply(lambda x: symbol_map[x])\nX = X.iloc[:, 1:].astype(\"uint8\").values\n\n## Prepare images: resize and put them in [0,1] range\nX_array = []\n\nfor i in tqdm(range(len(X))):\n\n    img = Im.fromarray(X[i, :].reshape((28,28))).resize((32, 32))\n    X_array.append((np.array(img) / 255).T)\n\nX_array = np.array(X_array)\n\n\n## Compute probabilities for input images\nif PRELOAD_INFERENCE_OUTPUTS:\n    outputs = pd.read_csv(\"../input/images/inference_output_emnist.csv\").iloc[:,1:]\nelse:\n    outputs = []\nxs = []\n\nfor label in tqdm(sorted(labels.unique())):\n\n    x = X_array[labels == label].reshape((-1,32,32,1))\n    l = labels[labels == label].values\n\n    x = x[:10000, :, :, :]\n    l = l[:10000]\n\n    xs.append(x)\n    \n    if not PRELOAD_INFERENCE_OUTPUTS:\n        try:\n            r = requests.post(\n                \"https://inference.fly.dev/score\", data = json.dumps({\"input\": x.tolist()})\n            )\n            if \"outputs\" in r.json():\n                outputs.append(pd.DataFrame(r.json()[\"outputs\"]).assign(label=l))\n            else:\n                raise ValueError(\"error...\")\n        except requests.exceptions.ConnectionError:\n            print(\"Connection problems. Contact the CTF organizers.\")\n\n\n## Prepare output\nif not PRELOAD_INFERENCE_OUTPUTS:\n    outputs = pd.concat(outputs).reset_index(drop=True)\nxss = np.concatenate(xs)","metadata":{"execution":{"iopub.status.busy":"2022-09-11T18:00:53.433943Z","iopub.execute_input":"2022-09-11T18:00:53.434353Z","iopub.status.idle":"2022-09-11T18:02:57.371471Z","shell.execute_reply.started":"2022-09-11T18:00:53.434320Z","shell.execute_reply":"2022-09-11T18:02:57.369941Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def stats(outputs, position, thr=0.999):\n    \n    out = outputs[\n        outputs.iloc[:, position] >= thr\n    ]\n    display(out.label.value_counts().head(3))\n    \ndef visuals(outputs, xss, position, thr=0.99):\n    out = outputs[\n        outputs.iloc[:, position] >= thr\n    ]\n    \n    avg_x = xss[out.index, :,:, 0].mean(axis=0)\n    \n    display(Im.fromarray((255 - avg_x * 255).astype(\"uint8\")))\n    \n","metadata":{"execution":{"iopub.status.busy":"2022-09-11T18:02:57.374188Z","iopub.execute_input":"2022-09-11T18:02:57.374730Z","iopub.status.idle":"2022-09-11T18:02:57.382642Z","shell.execute_reply.started":"2022-09-11T18:02:57.374683Z","shell.execute_reply":"2022-09-11T18:02:57.381579Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for i in range(6):\n    stats(outputs, i, 0.9997)","metadata":{"execution":{"iopub.status.busy":"2022-09-11T18:02:57.384500Z","iopub.execute_input":"2022-09-11T18:02:57.384878Z","iopub.status.idle":"2022-09-11T18:02:57.434160Z","shell.execute_reply.started":"2022-09-11T18:02:57.384844Z","shell.execute_reply":"2022-09-11T18:02:57.433076Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for i in range(6):\n    visuals(outputs, xss, i, 0.9997)","metadata":{"execution":{"iopub.status.busy":"2022-09-11T18:02:57.436283Z","iopub.execute_input":"2022-09-11T18:02:57.436618Z","iopub.status.idle":"2022-09-11T18:02:57.473060Z","shell.execute_reply.started":"2022-09-11T18:02:57.436589Z","shell.execute_reply":"2022-09-11T18:02:57.471985Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"try:\n    r = requests.post(\n        \"http://flags.fly.dev:5000/score\", \n        json=json.dumps({\"challenge_id\": \"inference\", \"submission\": \"FLAG{D3FC0N}\"})\n    )\n    if not r.text.endswith(\"Try again.\"):\n        print(r.text)\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T20:02:30.599077Z","iopub.execute_input":"2022-09-11T20:02:30.599541Z","iopub.status.idle":"2022-09-11T20:02:30.745902Z","shell.execute_reply.started":"2022-09-11T20:02:30.599499Z","shell.execute_reply":"2022-09-11T20:02:30.745076Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"inference\", \"2MNHG9C635NEC35AQ0E797JFWLSN43UKKUUIWD63LXKUB437QIW3XM8BLJGN6GVV6HS8DRAMB9QO2DYR82QDHFKKAO7DU}JHTNBBA6P1DUCJ{LYF4PA78W23MP7SQJWW\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T23:06:09.768995Z","iopub.execute_input":"2022-09-11T23:06:09.770364Z","iopub.status.idle":"2022-09-11T23:06:09.779878Z","shell.execute_reply.started":"2022-09-11T23:06:09.770324Z","shell.execute_reply":"2022-09-11T23:06:09.779020Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"deepfake\"></a>\n# Deepfake (1 challenge) ✅\n\nMy first try was to change the message of the video and/or the audio to **not be negative against dogs**. I used Adobe to cut the video in the parts where \"dog\" and \"cat\" were said and tried to swap them so that the video is not anymore negative against dogs, but cats. It took me a lot of time to get a reasonable video on this idea, and in the end it didn't work.\n\nMy next idea was to **cut all the mentions to dogs**, but it didn't work either.\n\nThese ideas were related to the negative message about dogs. However, thinking about the technical part, the deepfake, I finally tried to **cut the video into a section where no deepfake was visible**. A section that shows the real face. And that idea worked! Note that my cut is very short, it's almost a picture.\n\n**Solution:** Get a cut of the video where no deepfake is visible and only the real face appears.\n","metadata":{}},{"cell_type":"code","source":"with open(f\"../input/images/no_deep_fake.mp4\", \"rb\") as f:\n    data = f.read()\nb64_data = base64.urlsafe_b64encode(data).decode()\nin_data = json.dumps({\"input\": b64_data})\nr = requests.post(\"https://deepfake.fly.dev/score\", data=in_data)\nprint(r.text)","metadata":{"execution":{"iopub.status.busy":"2022-09-11T18:03:11.585737Z","iopub.execute_input":"2022-09-11T18:03:11.586180Z","iopub.status.idle":"2022-09-11T18:03:13.705788Z","shell.execute_reply.started":"2022-09-11T18:03:11.586141Z","shell.execute_reply":"2022-09-11T18:03:13.704503Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"deepfake\", \"PPGU9JZWU5L84O7GGCCF75KBXI25ZZTYTA3UZG35XLDWPA3JGRCA43ALT6A56DWRRCUUTD1BTUEOO}HOUCLOH5DQXZRIJI5EXVXGUBERRJQPTEDB2N{IHP0NGNAYLF8K\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T23:06:35.850002Z","iopub.execute_input":"2022-09-11T23:06:35.850713Z","iopub.status.idle":"2022-09-11T23:06:35.861464Z","shell.execute_reply.started":"2022-09-11T23:06:35.850668Z","shell.execute_reply":"2022-09-11T23:06:35.860214Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"adversarial\"></a>\n# Adversarial Attacks - Hotterdog, Salt & Theft (3 challenges) ✅✅✅\n\nI grouped these 3 challenges here because I used **adversarial attacks** and similar code to get their flags. However, as I am not an expert in deep learning, and especially not in adversarial attacks, I had several problems with these challenges. We will see also in next section that at the beginning I tried several unsuccessful ideas for hotterdog that did not use adversarial attacks.\n\nAfter some research reading about adversarial attacks, I got that the 3 challenges are **targeted adversarial attack** problems. For hotterdog we don't have the model, and for the other two we have the model, but for theft it is encoded (or encrypted, I did not discover it) and salt model modifies the input image before classification. From what I read, not having the model makes gradient-based attacks not useful, as you cannot compute the gradient of the original model. But at least for theft and salt, I could use them. A great reference on this is this medium article that was suggested in kaggle discussions: [Machine Learning is Fun Part 8: How to Intentionally Trick Neural Networks](https://medium.com/@ageitgey/machine-learning-is-fun-part-8-how-to-intentionally-trick-neural-networks-b55da32b7196).\nI use a modified version of the code in that article for the challenges.\n\nTheoretically, hotterdog could not be solved with this method, but reading about **targeted black-box adversarial attacks** you need also de probability of the target label, which we don't have for this problem.  As several kagglers solved it, I thought it is doable with the previous gradient-based attacks, so I tried with pretrained models instead. I tried with all pretrained keras models with the same input sizes, but `VGG16` and `MobileNet` were the most helpful. Although the predictions were not exactly the same, making one of those models predict the images as something else, made hotterdog endpoint change the label, and in the end we got the hotdog!\n\nGiven the success, I did not try decoding theft pickle, and tried with the same pretrained models. This challenge was easier, especially because we have the top-5 predictions as output of the endpoint, instead of the top-1 as with hotterdog. And for salt we used the model in the input set instead of the pretrained models.\n\nThe code below is the code we used for hotterdog. It is not run now as it took some time and different runs as we explain in the next section. Also, I marked with comments \"## <--\" the parts that have to be changed for each problem, which are: the pretrained (or loaded) model, the target class, the original image to modify, and the challenge endpoint to get feedback about progress. \n\n**Note:** The original notebook where I run the code for these challenges uses GPU to get the adversarial image faster.","metadata":{}},{"cell_type":"code","source":"EXECUTE_HOTTERDOG = False\n\nif EXECUTE_HOTTERDOG:\n    import numpy as np\n    from keras.preprocessing import image\n    from keras.applications import mobilenet as nn\n    from keras import backend as K\n    from PIL import Image\n    from pprint import pprint\n\n    import tensorflow as tf\n    tf.compat.v1.disable_eager_execution()\n\n    # Load pre-trained image recognition model\n    model = nn.MobileNet(weights=\"imagenet\")         ## <-- Model\n\n    # Grab a reference to the first and last layer of the neural net\n    model_input_layer = model.layers[0].input\n    model_output_layer = model.layers[-1].output\n\n    # Choose an ImageNet object to fake\n    # The list of classes is available here: https://gist.github.com/ageitgey/4e1342c10a71981d0b491e1b8227328b\n    # Class #934 is \"hotdog\"\n    object_type_to_fake = 934         ## <-- Target class\n\n    # Load the image to hack\n    img = image.load_img(f\"../input/ai-village-ctf/hotterdog/chester.png\", target_size=(224, 224))         ## <-- Original image\n    original_image = image.img_to_array(img)\n\n    # Scale the image so all pixel intensities are between [-1, 1] as the model expects\n    original_image /= 255.\n    original_image -= 0.5\n    original_image *= 2.\n\n    # Add a 4th dimension for batch size (as Keras expects)\n    original_image = np.expand_dims(original_image, axis=0)\n\n    # Pre-calculate the maximum change we will allow to the image\n    # We'll make sure our hacked image never goes past this so it doesn't look funny.\n    # A larger number produces an image faster but risks more distortion.\n    max_change_above = original_image + 0.01\n    max_change_below = original_image - 0.01\n\n    # Create a copy of the input image to hack on\n    hacked_image = np.copy(original_image)\n\n    # How much to update the hacked image in each iteration\n    learning_rate = 0.3\n\n    # Define the cost function.\n    # Our 'cost' will be the likelihood out image is the target class according to the pre-trained model\n    cost_function = model_output_layer[0, object_type_to_fake]\n\n    # We'll ask Keras to calculate the gradient based on the input image and the currently predicted class\n    # In this case, referring to \"model_input_layer\" will give us back image we are hacking.\n    gradient_function = K.gradients(cost_function, model_input_layer)[0]\n\n    # Create a Keras function that we can call to calculate the current cost and gradient\n    grab_cost_and_gradients_from_model = K.function([model_input_layer, K.learning_phase()], [cost_function, gradient_function])\n\n    cost = 0.0\n\n    num_iters = 0\n    # In a loop, keep adjusting the hacked image slightly so that it tricks the model more and more\n    # until it gets to at least 80% confidence\n\n    finished = False\n    print_cost = 0\n    while not finished:\n\n        num_iters += 1\n\n        # Check how close the image is to our target class and grab the gradients we\n        # can use to push it one more step in that direction.\n        # Note: It's really important to pass in '0' for the Keras learning mode here!\n        # Keras layers behave differently in prediction vs. train modes!\n        cost, gradients = grab_cost_and_gradients_from_model([hacked_image, 0])\n\n        # Move the hacked image one step further towards fooling the model\n        hacked_image += gradients * learning_rate\n\n        # Ensure that the image doesn't ever change too much to either look funny or to become an invalid image\n        hacked_image = np.clip(hacked_image, max_change_below, max_change_above)\n        hacked_image = np.clip(hacked_image, -1.0, 1.0)\n\n        if (cost * 100 >= print_cost) :\n\n            print_cost = (cost * 100) * 1.1\n\n            # Direct cost\n            print(\"Model's predicted likelihood that the image is a hotdog: {:.8}%\".format(cost * 100))\n\n            # Top 5 preds of model\n            preds = model.predict(hacked_image)\n            pprint(nn.decode_predictions(preds, top=10)[0])\n\n            # Hotdog API prediction\n            h_img = Image.fromarray(\n                ((hacked_image[0] / 2 + 0.5) * 255).astype(np.uint8)\n            )\n            img_bytes = h_img.tobytes()\n            b64_img_bytes = base64.urlsafe_b64encode(img_bytes)\n\n            try:          ## <-- Chalenge endpoint\n                r = requests.post(\"https://hotdog.fly.dev/score\", data=json.dumps({\"input\": b64_img_bytes.decode()}))\n                print(r.text)\n            except requests.exceptions.ConnectionError:\n                print(\"Connection problems. Contact the CTF organizers.\")\n\n            print(\"-------------\")\n\n\n    # De-scale the image's pixels from [-1, 1] back to the [0, 255] range\n    img = hacked_image[0]\n    img /= 2.\n    img += 0.5\n    img *= 255.\n\n    # Save the hacked image!\n    im = Image.fromarray(img.astype(np.uint8))\n    im.save(\"hacked-image.png\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T18:06:39.016643Z","iopub.execute_input":"2022-09-11T18:06:39.017934Z","iopub.status.idle":"2022-09-11T18:06:39.045618Z","shell.execute_reply.started":"2022-09-11T18:06:39.017875Z","shell.execute_reply":"2022-09-11T18:06:39.044611Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"hotterdog\"></a>\n## Hotterdog\n\nBefore trying adversarial attacks, I tried similar strategies as with hotdog, because I thought they were connected. I downloaded some images from the Internet without any success.. I **first tried with wiener dogs dressed up as hotdogs**. Below you can see some of them.","metadata":{}},{"cell_type":"code","source":"IMAGE_DIMS = (224,224)\ndisplay(Im.open(f\"../input/images/chester_hotdog1.jpeg\").resize(IMAGE_DIMS))\ndisplay(Im.open(f\"../input/images/chester_hotdog5.jpeg\").resize(IMAGE_DIMS))\ndisplay(Im.open(f\"../input/images/chester_hotdog6.jpeg\").resize(IMAGE_DIMS))","metadata":{"execution":{"iopub.status.busy":"2022-09-11T18:07:22.735922Z","iopub.execute_input":"2022-09-11T18:07:22.736437Z","iopub.status.idle":"2022-09-11T18:07:22.841616Z","shell.execute_reply.started":"2022-09-11T18:07:22.736397Z","shell.execute_reply":"2022-09-11T18:07:22.840342Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"As the original image had not to be changed significantly, I thought that **adding hotdogs to the original picture** would work. Below you can see my attempts, that were again unsuccessful.","metadata":{}},{"cell_type":"code","source":"display(Im.open(f\"../input/images/chester-overlay_7.png\").resize(IMAGE_DIMS))\ndisplay(Im.open(f\"../input/images/chester-overlay_10.png\").resize(IMAGE_DIMS))","metadata":{"execution":{"iopub.status.busy":"2022-09-11T18:07:32.658891Z","iopub.execute_input":"2022-09-11T18:07:32.659701Z","iopub.status.idle":"2022-09-11T18:07:33.041902Z","shell.execute_reply.started":"2022-09-11T18:07:32.659649Z","shell.execute_reply":"2022-09-11T18:07:33.040700Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I tried those approaches before learning about **adversarial attacks**, but then I tried directly with that. I started with `VGG16` as its output seemed closer to the endpoint model. However, the probability of hotdog was so low, that the model was very slow at the beginning. Instead, I tried with different classes close to hotdog, but far from dog breeds. I first used object classes in the kitchen, and then I tried with **cheeseburger, french loaf, and bagel**. The probability of all of these classes was at the top of the predictions for `VGG16`, but the challenge endpoint was not predicting exactly them (I remember shower curtain for instance), but at least it was not predicting dog breeds anymore.\n\nBut then I changed to `MobileNet` using the adversarial image form `VGG16` and it worked like a charm . I first modified the image to cheeseburger, and the endpoint was predicting it correctly, and then hotdog, and I got the flag.\n\n**Solution:** Generate an adversarial image with `VGG16` and `MobileNet` as explained above.","metadata":{}},{"cell_type":"code","source":"IMAGE_DIMS = (224,224)\nimg = Im.open(f\"../input/images/chester-hotterdog.png\")\nimg = img.resize(IMAGE_DIMS)\n\ndisplay(img)\n\nimg_bytes = img.tobytes()\nb64_img_bytes = base64.urlsafe_b64encode(img_bytes)\n\ntry:\n    r = requests.post(\"https://hotterdog.fly.dev/score\", data=json.dumps({\"input\": b64_img_bytes.decode()}))\n    print(r.text)\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T18:09:59.278405Z","iopub.execute_input":"2022-09-11T18:09:59.279055Z","iopub.status.idle":"2022-09-11T18:10:00.200309Z","shell.execute_reply.started":"2022-09-11T18:09:59.279010Z","shell.execute_reply":"2022-09-11T18:10:00.199016Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"hotterdog\", \"7RKQYVO1HY57694OOC632TL5HIOELDA2S{CVLLMG07GRD8QBZYMD3TBSNHZDOXQK}QH2Y37OWSHEPDGT2LH0GUZPUJGTAA46G36V38DOO6KOLLJIOTQOS3IJNVKXW1FH\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T23:07:04.702515Z","iopub.execute_input":"2022-09-11T23:07:04.703203Z","iopub.status.idle":"2022-09-11T23:07:04.713730Z","shell.execute_reply.started":"2022-09-11T23:07:04.703156Z","shell.execute_reply":"2022-09-11T23:07:04.712452Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"theft\"></a>\n## Theft\n\nHere I also tried with `VGG16` first, and here I didn't have to go through intermediate classes first. Again, `VGG16` predicted loggerhead with high confidence, but the endpoint didn't. Changing to `MobileNet` made it work again.\n\n**Solution:** Generate an adversarial image with `VGG16` and `MobileNet`. ","metadata":{}},{"cell_type":"code","source":"IMAGE_DIMS = (224,224)\nimg = Im.open(f\"../input/images/owl_to_turtle.png\")\nimg = img.resize(IMAGE_DIMS)\n\ndisplay(img)\n\nimg_bytes = img.tobytes()\nb64_img_bytes = base64.urlsafe_b64encode(img_bytes)\n\ntry:\n    r = requests.post(\"https://theft.fly.dev/score\", data=json.dumps({\"input\": b64_img_bytes.decode()}))\n    print(r.text)\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T18:10:11.953763Z","iopub.execute_input":"2022-09-11T18:10:11.954492Z","iopub.status.idle":"2022-09-11T18:10:12.259732Z","shell.execute_reply.started":"2022-09-11T18:10:11.954456Z","shell.execute_reply":"2022-09-11T18:10:12.258506Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"theft\", \"R4LETL3PACHENLC01P4H2}7WE9B?02TT4RF6Y83QU70BJ3A6O00KVV10NAK51X2LKTVGAYYJNWE{64OJNZYMV3A0GFNCECYC1GD4SGS2D1AP33AOLL3XR5GSVRQ7VP2P\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T23:07:20.467842Z","iopub.execute_input":"2022-09-11T23:07:20.468277Z","iopub.status.idle":"2022-09-11T23:07:20.479027Z","shell.execute_reply.started":"2022-09-11T23:07:20.468243Z","shell.execute_reply":"2022-09-11T23:07:20.477455Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"salt\"></a>\n## Salt\n\nThis one was similar to hotterdog. Although we had the model, and I thought it included the salting in the first layers, it did not. The output from the model and the endpoint differed. Anyway, I used the provided model.\n\nAs with hotterdog, the probability of owl was low and the process started slowly. I used the same strategy. I first put as target bird classes in the top predicted classes. We started with **albatross**, and then moved to **bustard, peacock, dowitcher, quail, weevil**, and finally **great grey owl**.\n\n**Note:** Before using the code above, we need to change tensorflow and keras versions to load the model as suggested in [this discussion.](https://www.kaggle.com/competitions/ai-village-ctf/discussion/346173)\n\n**Solution:** Generate an adversarial image as explained above with the model provided by the chalenge. ","metadata":{}},{"cell_type":"code","source":"IMAGE_DIMS = (224,224)\nimg = Im.open(f\"../input/images/turtle_to_owl.png\")\nimg = img.resize(IMAGE_DIMS)\n\ndisplay(img)\n\nimg_bytes = img.tobytes()\nb64_img_bytes = base64.urlsafe_b64encode(img_bytes)\n\ntry:\n    r = requests.post(\"https://salt.fly.dev/score\", data=json.dumps({\"input\": b64_img_bytes.decode()}))\n    print(r.text)\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T23:07:33.660074Z","iopub.execute_input":"2022-09-11T23:07:33.660897Z","iopub.status.idle":"2022-09-11T23:07:34.518714Z","shell.execute_reply.started":"2022-09-11T23:07:33.660851Z","shell.execute_reply":"2022-09-11T23:07:34.517297Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"salt\", \"3NNG2ZJBP3A4TZL9Y0KTIYL9MI<RKYSULSWEK88G2B{ATGJK60CJSBO0814Z60X2}R394WZ9UZSNUL190FXGLL7W461F81TZAIFBFKIK3BIDYXTP4TIEZ04T1VPGCROV\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T23:07:47.531872Z","iopub.execute_input":"2022-09-11T23:07:47.532678Z","iopub.status.idle":"2022-09-11T23:07:47.544795Z","shell.execute_reply.started":"2022-09-11T23:07:47.532624Z","shell.execute_reply":"2022-09-11T23:07:47.543106Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"crop1\"></a>\n# Crop1 (1 challenge) ✅\n\nAs at first sight the model functions were too complex and a little bit cryptic I decided to start with a different approach than reading and understanding the provided functions. As we have the code of the model locally, and we can run experiments without sending requests to the endpoint I decided tro **try with genetic algorithms**.","metadata":{}},{"cell_type":"code","source":"model = joblib.load(f\"{dir_name}/crop/04JUN_crop_model.pckl\")\nX_comp = joblib.load(f\"{dir_name}/crop/X_comp.pckl\")\n\n# Helper Functions\ndef get_ij(idx):\n    idx = 8-idx\n    j = idx//3\n    i = 2-idx%3\n    return i, j\n\ndef resize(X, size=(224,224)):\n    X_new = np.array(Im.fromarray(X.astype(np.uint8)).resize(size, resample=3))\n    return X_new\n\ndef get_cropped(X, idx):\n    n = 75*3\n    i, j = get_ij(idx)\n    crop = resize(X,(n,n))[j*75:(j+1)*75,i*75:(i+1)*75,:]\n    return crop\n\ndef i2t(img:np.array):\n    return np.array(img, dtype=np.float32)/255.\n\ndef make_X(x, size=(224,224)):\n    side = int((len(x)//3)**0.5)\n    x = np.where(x<255,x,255)\n    x = np.where(x>0,x,0)\n    X_np = x.astype(int).reshape([side, side, 3])\n    X_np = np.array(Im.fromarray(X_np.astype(np.uint8)).resize(size, resample=Im.Resampling.NEAREST))\n    return X_np\n\ndef pt_ft(X_np):\n    return torch.FloatTensor(X_np)\n\ndef score(x, model=model, X_comp = np.divide(X_comp,0b1010), debug=False):\n    x = (x).astype(int)\n    model.eval()\n    X = make_X(x)\n    preds = model(pt_ft(i2t(X.T[None])))\n    idx = preds.argmax().item()\n    expected = ((25.5-X_comp)).flatten()\n    actual_square = get_cropped(X, idx)\n    actual = actual_square.flatten().astype(int)\n    if debug:\n        print({\"idx\":idx})\n        plt.imshow(actual_square)\n        plt.show()\n    sse = ((actual-(expected*10).astype(int))**2).sum()\n    redness = (actual_square[:,:,0][actual_square[:,:,0]>230]**2).sum()+1\n    if idx==8:\n        return (1+preds.max().item())*sse*redness\n    return sse*redness\n\ndef get_submittable(x):\n    return \",\".join([str(int(n)) for n in x])\n\ncfn = lambda x: [f\"background-color: RGB({x.R},{x.G},{x.B})\"] * 3\ndef show_colors(ans):\n    ans = [int(x) for x in ans.split(\",\")]\n    squares = pd.DataFrame(np.array(ans[:27]).reshape([9,3]), columns=[\"R\",\"G\",\"B\"]).T.style.apply(cfn, axis=0)\n    circles = pd.DataFrame(np.array(ans[27:]).reshape([9,3]), columns=[\"R\",\"G\",\"B\"]).T.style.apply(cfn, axis=0)\n    display(squares)\n    display(circles)","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-09-11T23:26:39.249822Z","iopub.execute_input":"2022-09-11T23:26:39.250371Z","iopub.status.idle":"2022-09-11T23:26:39.646311Z","shell.execute_reply.started":"2022-09-11T23:26:39.250322Z","shell.execute_reply":"2022-09-11T23:26:39.645361Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"PRELOAD_GENETIC = True\n\nif PRELOAD_GENETIC:\n    X = np.load(\"../input/images/crop1_genetic_solution_3.npy\")\nelse:\n\n    !pip install PyGAD\n\n    import pygad\n    from tqdm import tqdm\n\n    def fitness_fun(X, idx):\n\n        X = (255 * X).astype(int)\n        im = np.array(get_submittable(X).split(\",\"))\n        return -score(im) / 1e7\n\n    SIZE = 10\n\n    num_generations = 1000\n    with tqdm(total=num_generations) as pbar:\n\n        ga_instance = pygad.GA(\n            num_generations=num_generations,\n            num_parents_mating=4,\n            fitness_func=fitness_fun,\n            sol_per_pop=20,\n            num_genes=SIZE**2*3,\n            init_range_low=0.0,\n            init_range_high=1.0,\n            mutation_percent_genes=0.1,\n            mutation_type=\"random\",\n            mutation_by_replacement=True,\n            random_mutation_min_val=0.0,\n            random_mutation_max_val=1.0,\n            on_generation=lambda x: pbar.update(1),\n            on_fitness=lambda x,y: print(max(y))\n        )\n        ga_instance.run()\n        \n    # Get best image\n    bb = ga_instance.best_solution()[0]\n    X = (bb.reshape((SIZE, SIZE, 3)) * 255).astype(int)\n    \n_ = plt.imshow(X)","metadata":{"execution":{"iopub.status.busy":"2022-09-11T23:55:03.666047Z","iopub.execute_input":"2022-09-11T23:55:03.667242Z","iopub.status.idle":"2022-09-11T23:55:03.802951Z","shell.execute_reply.started":"2022-09-11T23:55:03.667196Z","shell.execute_reply":"2022-09-11T23:55:03.801994Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"im = np.array(get_submittable(X.flatten()).split(\",\"))\nscore(im, debug=True) / 1e7","metadata":{"execution":{"iopub.status.busy":"2022-09-11T23:55:06.361604Z","iopub.execute_input":"2022-09-11T23:55:06.362010Z","iopub.status.idle":"2022-09-11T23:55:06.669778Z","shell.execute_reply.started":"2022-09-11T23:55:06.361978Z","shell.execute_reply":"2022-09-11T23:55:06.668773Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We can see that the solution is far from the target, but in the original notebook I got an image scoring 1.06e7. Although I did not get the flag with the genetic solution, I got a good image from which I can now iterate. For instace, we observe that **the solution has noise in all the ids except for the id 1**, for which we observe some structure, so it will suffice to modify that part of the image. Also, now we can **try to understand the score code with this example** which makes things easier. \n\nMy second approach consisted on modifying the genetic image to get the flag. By going through the score code with the image it was clear that the image gets resized to a 224x224 image and compared to a `expected` image built from `X_comp` to get the `sse` component of the final score. So we **put directly the expected image in index 1 of the resized image**. \nHowever, as the `expected` image has red pixels with intensity greater than 230, **instead of multyplying by 10, we multiply by 9 to reduce the `rednes` component of the cropped image**.\n\n**Solution:** Use genetic algorithms to get an image that gets assigned to a given id (here 0) and has a noise background. Change the cropped image of the resized version by a less red version of the `expected` image.","metadata":{}},{"cell_type":"code","source":"# Compute and plot expected\nexpected = 9* (25.5-np.divide(X_comp, 0b1010)).astype(\"uint8\")\n_ = plt.imshow(expected)","metadata":{"execution":{"iopub.status.busy":"2022-09-11T23:57:22.083210Z","iopub.execute_input":"2022-09-11T23:57:22.083631Z","iopub.status.idle":"2022-09-11T23:57:22.284728Z","shell.execute_reply.started":"2022-09-11T23:57:22.083599Z","shell.execute_reply":"2022-09-11T23:57:22.283220Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Get resized image with expected\nresized_X = make_X(im.astype(int))\nresized_X[149:,:75,:] = expected\n_ = plt.imshow(resized_X)","metadata":{"execution":{"iopub.status.busy":"2022-09-11T23:57:43.270952Z","iopub.execute_input":"2022-09-11T23:57:43.271443Z","iopub.status.idle":"2022-09-11T23:57:43.493555Z","shell.execute_reply.started":"2022-09-11T23:57:43.271408Z","shell.execute_reply":"2022-09-11T23:57:43.492226Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"im = np.array(get_submittable(resized_X.flatten()).split(\",\"))\nscore(im, debug=True) / 1e7","metadata":{"execution":{"iopub.status.busy":"2022-09-11T23:57:46.339734Z","iopub.execute_input":"2022-09-11T23:57:46.340522Z","iopub.status.idle":"2022-09-11T23:57:46.790519Z","shell.execute_reply.started":"2022-09-11T23:57:46.340481Z","shell.execute_reply":"2022-09-11T23:57:46.789318Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"try:\n    r = requests.post(\"https://crop1.fly.dev/score\", data=json.dumps({\"data\": get_submittable(resized_X.flatten())}))\n    print(r.text)\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-12T00:00:00.883119Z","iopub.execute_input":"2022-09-12T00:00:00.883921Z","iopub.status.idle":"2022-09-12T00:00:01.915992Z","shell.execute_reply.started":"2022-09-12T00:00:00.883875Z","shell.execute_reply":"2022-09-12T00:00:01.914419Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"crop1\", \"QSNLOZC6{T0QH2On}4ZJC9CHWZnCNTMLFPTCA7Je1LEB9B1V2gS96UNVWVAKR1OM7IFX2ChZADRES2G0QWJTBr6aCDL2MTQCH0LC5UQY6WVFNTBagQUQNHHGUFZMH540\")","metadata":{"execution":{"iopub.status.busy":"2022-09-12T00:00:26.612103Z","iopub.execute_input":"2022-09-12T00:00:26.612966Z","iopub.status.idle":"2022-09-12T00:00:26.637161Z","shell.execute_reply.started":"2022-09-12T00:00:26.612915Z","shell.execute_reply":"2022-09-12T00:00:26.632273Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"waf\"></a>\n# WAF (1 challenge) ✅\n\nFor this challenge I followed several hints suggested in discord. The first one was that the provided input `\"mFzaC==\"` is not base64, but should be part of the solution. So **I tried adding base64 chars before the string and explored the decoded strings**. One of the words below is `bash` so the attack should use the bash command.","metadata":{}},{"cell_type":"code","source":"import base64\nimport string\n\nwords = set()\nfor c in string.ascii_letters + string.digits:\n    words.add(base64.b64decode(f\"{c}mFzaC==\"))\nwords","metadata":{"execution":{"iopub.status.busy":"2022-09-11T18:46:39.840085Z","iopub.execute_input":"2022-09-11T18:46:39.840588Z","iopub.status.idle":"2022-09-11T18:46:39.850886Z","shell.execute_reply.started":"2022-09-11T18:46:39.840554Z","shell.execute_reply":"2022-09-11T18:46:39.849684Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I searched for attacks with the bash command, but I didn't find anything that worked. **The next hints I used were these ones that `@moo` posted in discord, the first one about [usual command line attacks](https://discord.com/channels/714203099698430053/1007383403680780379/1010897638884900905) and the second one about [encoding spaces](https://discord.com/channels/714203099698430053/1007383403680780379/1010898541020987392)**.\n\nSo I tried different things with the bash command and found that writing it as `\\bin\\bash` with a preceding space we got a malicious requested detected with lower score. I focused only on command line attacks and after searching about different attacks I finally found the [ShellShocker attack](https://blog.sucuri.net/2014/09/bash-shellshocker-attacks-increase-in-the-wild-day-1.html) which is the zero-day of this challenge. However, we need to add and space at the beginning to make it pass as undetected. Below you can see the two options with and without the space.\n\n**Solution:** Use the shellshock attack `() { :;}; /bin/bash -c...` with a preceding space.","metadata":{}},{"cell_type":"code","source":"import base64\n\nb = base64.b64encode(b\"() { :;}; /bin/bash -c\")\n\ninput_ = b.decode(\"utf-8\")\ndata = json.dumps(\n    {\"input\": input_}\n)\n\ntry:\n    r = requests.post(\"https://waf.fly.dev/score\", data=data)\n    print(r.json())\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T18:46:50.919737Z","iopub.execute_input":"2022-09-11T18:46:50.920142Z","iopub.status.idle":"2022-09-11T18:46:51.085863Z","shell.execute_reply.started":"2022-09-11T18:46:50.920107Z","shell.execute_reply":"2022-09-11T18:46:51.084632Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"b = base64.b64encode(b\" () { :;}; /bin/bash -c\")\n\ninput_ = b.decode(\"utf-8\")\ndata = json.dumps(\n    {\"input\": input_}\n)\n\ntry:\n    r = requests.post(\"https://waf.fly.dev/score\", data=data)\n    print(r.json())\nexcept requests.exceptions.ConnectionError:\n    print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T18:46:56.261984Z","iopub.execute_input":"2022-09-11T18:46:56.262391Z","iopub.status.idle":"2022-09-11T18:46:56.439052Z","shell.execute_reply.started":"2022-09-11T18:46:56.262358Z","shell.execute_reply":"2022-09-11T18:46:56.437754Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"waf\", \"L88IKCP45EXPROTKQ0ZSHWRVCJRKHQDU2BSG9HNU0DQ0FLF7NIT9HFR3FRLR6{G5NPGGE2GEYF04GINAUF64L1V4H2F0RNKGFQA}197188P3ZXKOOWTFZ6OT3IBYZYQF\")","metadata":{"execution":{"iopub.status.busy":"2022-09-11T23:08:21.033308Z","iopub.execute_input":"2022-09-11T23:08:21.034258Z","iopub.status.idle":"2022-09-11T23:08:21.045456Z","shell.execute_reply.started":"2022-09-11T23:08:21.034215Z","shell.execute_reply":"2022-09-11T23:08:21.044213Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"secret-sloth\"></a>\n# Secret sloth (1 challenge) ❌\n\nI spent the most time with this challenge and I could not solve it. I researched several steganography and watermarking techniques, but none work.\n\nAt the beginning **I tried by looking at the structure of the image**: the bytes, the PNG chunks (there was a matplotlib reference in the tEXt chunk), the strings, the bitmaps. For this I used some online tools such as [StegOnline](https://stegonline.georgeom.net/upload). Nothing worked.\n\nI searched for the original images two and I found it in two websites, one in [reddit](https://www.reddit.com/r/Slothswithhats/comments/1cpj4v/he_killed_a_man_for_that_hat/), but comparing its structure was not useful either. I found that the image histograms of the 3 channels for the stego image had a weird peak at 149 that was not present in the original image, but I could not get anything from that. \n\n**My second approach was to get a visible message using linear algebra and signal processing transformations** as suggested by `@moo` [in discord](https://discord.com/channels/714203099698430053/1007383403680780379/1011520310056583189). I tried with different filters to detect edges, **DCT, DFT, DWT, PCA, SVD**, but I couldn't get the flag. I tried to apply those transformations to the grayscale image and the different channels. I even converted the image to different color models and applied the transformation to the new channels. Nothing worked, but I will explain the closest I got to guessing some string structure.\n\n**Closest idea:** By fitting PCA on the original image (reddit image) and projecting the original and the stego image to that space and looking at the difference. The best visible result applied those transformations to the red channel. As you can see in the code below we can guess some chars in the low part of the image, but there is no clear message there.","metadata":{}},{"cell_type":"code","source":"from sklearn.decomposition import PCA\n\nimgarr = np.array(Im.open(f'{dir_name}/secret.sloth/secret-sloth.png').getchannel(\"R\")) /255\nimgarr2 = np.array(Im.open(f'../input/images/original_sloth3.png').getchannel(\"R\")) /255\n\nm = PCA()\nm.fit(imgarr2)\n\n_ = plt.imshow(\n    m.transform(imgarr) - m.transform(imgarr2),\n    cmap=\"viridis\"\n)","metadata":{"execution":{"iopub.status.busy":"2022-09-11T19:14:06.649446Z","iopub.execute_input":"2022-09-11T19:14:06.649856Z","iopub.status.idle":"2022-09-11T19:14:07.636558Z","shell.execute_reply.started":"2022-09-11T19:14:06.649821Z","shell.execute_reply":"2022-09-11T19:14:07.635477Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"_ = plt.imshow(\n    (m.transform(imgarr) - m.transform(imgarr2))[275:290,150:],\n    cmap=\"viridis\"\n)","metadata":{"execution":{"iopub.status.busy":"2022-09-11T19:14:11.428665Z","iopub.execute_input":"2022-09-11T19:14:11.429111Z","iopub.status.idle":"2022-09-11T19:14:11.627980Z","shell.execute_reply.started":"2022-09-11T19:14:11.429072Z","shell.execute_reply":"2022-09-11T19:14:11.626850Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"crop2\"></a>\n# Crop2 (1 challenge) ❌\n\nI did not spend too much in this challenge. I put all my efforts on secret sloth given that people solved it and not crop2. I can only share the two main hypotheses that I wanted to try from what I read about the LeViT transformer. Again, I am not an expert in deep learning and these ideas might be wavy and incorrect.\n\n**Hypotheses:**\n\n- The transformer should have a path from input to output that isolates the poisoned image and predicts a totally very different output. The activation functions through that path only allow the poisoned image pass, and no other images that get a prediction through different paths.\n\n- From what I understood, LeViT has a supervised classifier and a distiller one that is trained with a RestNet output. So maybe, during the training process, the distiller was trained only with the poisoned image and the wrong output and the supervised classifier was trained with the correct model (or the other way). As the final prediction is the average of the two outputs, if they are in different scales one might be overriding the others outputs. So my second hypothesis is that output1 overrides output2 for the general images but the poisoned one, for which output2 overrides output1 and that way we get the poisoned classification.","metadata":{}},{"cell_type":"markdown","source":"<a id=\"conclusions\"></a>\n# Conclusions\n\nThis was my first CTF challenge, and apart from learning a lot of new concepts and techniques, I had a lot of fun learning and solving the challenges. I am so thankful to AI Village and the organizers for the competition. It is very different to the usual kaggle competitions and I hope this is the beginning of many more CTF competitions!","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## ","metadata":{}}]}