{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Intro/ My thoughts on the competition\n\nSince I never have time at DEFCON to do CTFs imagine my surprise when I saw a Kaggle competition post conference. Inconceivable. Except I had to re-certify a few things, catch up on my PTO, be a father... this is fine.\n\n\n\n\n\nI have been in Cyber Security for over 16 years but with Machine Learning as a hobby for the past 4... and it shows. Many of the challenges I struggled with because I practiced with only a narrow set of tools in the past. Most I understood the gist of it but could not make that important implementation step to get the flag. This CTF set me up to think about different protocols and really widen my practical skillsets.\n\n## Summary\nGood stuff, do it, chill out with the [Tron soundtrack](https://www.youtube.com/watch?v=F4eccPBFEjE). \n\nTo help you find what you might be looking I split this notebook into a few different sections.\n\n1. [Lessons learned](#lessons)\n\n2. How I solved the following challenges:\n* [Hotdog](#hotdog)\n* [Math 1](#math1), [2-4](#math2)\n* [Honorstudent](#Honor)\n* [Bad to Good](#Bad2Good)\n* [Baseball](#Baseball)\n* [FORENSICS](#Forensics)\n* [Token](#Token)\n* [DEEPFAKE](#deepfake)\n* [Murderbots](#Murderbots)\n\n\n3. What I tried on the following challenges:\n* [WIFI](#wifi)\n* [HOTTERDOG](#hotterdog)\n* [Secret Sloth](#secretsloth)\n* [Inference](#Inference)\n* [WAF](#WAF)\n* [Excuse me your training data](#Excuse)\n* [THEFT](#Theft)\n* [SALT](#salt)\n* [Crop1](#crop1)\n* [Crop2](#crop2)\n\n\n","metadata":{}},{"cell_type":"markdown","source":"# Stuff to import","metadata":{}},{"cell_type":"markdown","source":"\n# Lessons Learned\nOverall lessons that are not on a specific challenge.\n\n## 1. **ALWAYS Publish:** \nIf you did this challenge... polish and publish your notebook. Next month, year, or decade you will probably need to borrow some of this code. Make sure you have a refence point. Several times I went back and borrowed old code for this notebook. Plus other people might get some good ideas from it... I am looking forward to Hotterdog myself.\n\n## 2. Build functions early; Every 2 hours go back and clean code\nWhen using notebooks code can get worse and worse as you try new solutions. Several times I found I built something but had to rearrange everything so that it would work on a run through. It also gave me the opportunity to make sure I understood what was happening with the output. \n\n## 3. Dont anticipate output and save weird output\nSeveral times I had gotten to flag only to figure out that I had already submitted that response. So I now make sure to code in what im looking for, what im NOT looking for, and if there is something else to put that on screen. \n\n## 4. Take the time to seperate problems\nIf you are doing everything in one large notebook you run the risk of messing up variables or taking things from past ones. Also, several times I had broken the Kaggle platform or kid fingers had found their way to trying to delete my hardwork. \n\n\n## 5. Brute Force: 60% of the time it works everytime\n\n\n\nTo start understanding the problem you need to really get in there and try a few different options. Then sometimes, a problem is just really easy to bruteforce (looking at you math). If you start developing a solution that can then be bruteforced it can help you get more information to see different trends as data is altered. Now with that added information you can reduce your bruteforce from some abstract big O notation down to a small range. \n","metadata":{}},{"cell_type":"code","source":"# We're just importing some stuff to make sure the challenges and helper code works.\n# Feel free to import anything else you need (or solve in a different language), but if you overwrite these, provided code may not work.\n\nimport pandas as pd\nimport numpy as np\nimport os\nimport requests\nfrom IPython.display import Image as Ima\nfrom IPython.display import clear_output, display\nfrom PIL import Image as Im\nfrom torch.utils.data import Dataset, DataLoader\nimport torch\nimport joblib\nimport matplotlib.pyplot as plt\nimport json\nimport base64\nimport shutil\nimport warnings\nimport copy\nwarnings.filterwarnings(\"ignore\")\nimport pickle\n\n\ndir_name = \"/kaggle/input/ai-village-ctf\"\n\n# Input data files are available in the read-only \"../input/\" directory\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session\n# A helper function for tracking flags. You could do it yourself, but why?\n\ndef add_flag(challenge_id: str, flag: str) -> str:\n try:\n submission_path = \"/kaggle/working/submission.csv\"\n template_path = f\"{dir_name}/submission.csv\"\n if not os.path.isfile(submission_path) :\n shutil.copyfile(template_path, submission_path)\n flags = pd.read_csv(submission_path, index_col=\"challenge_id\")\n valid = flags.index.to_list()\n if challenge_id in valid:\n flags.at[challenge_id] = flag\n flags.to_csv(submission_path)\n print(\"Flag saved. If you want to update the scoreboard, click 'submit' on the right. Remember that you can only update your score on the scoreboard 50 times per day\")\n else:\n print(\"Invalid challenge_id\")\n except:\n print(\"We had trouble saving that flag. Keep a copy of it and contact the CTF organizers.\")\n \n!ls /kaggle/input/ai-village-ctf/","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-09-16T21:40:03.618458Z","iopub.execute_input":"2022-09-16T21:40:03.618965Z","iopub.status.idle":"2022-09-16T21:40:04.720947Z","shell.execute_reply.started":"2022-09-16T21:40:03.618924Z","shell.execute_reply":"2022-09-16T21:40:04.719278Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 2. Solutions I solved! \n* [Hotdog](#hotdog)\n* [Math 1](#math1), [2-4](#math2)\n* [Honorstudent](#Honor)\n* [Bad to Good](#Bad2Good)\n* [Baseball](#Baseball)\n* [FORENSICS](#Forensics)\n* [Token](#Token)\n* [DEEPFAKE](#deepfake)\n* [Murderbots](#Murderbots)","metadata":{}},{"cell_type":"markdown","source":"\n# HOTDOG (id: hotdog) -- 200 Points\n\nI just gave it a picture of a hotdog. ","metadata":{}},{"cell_type":"code","source":"#Ima(filename=f'{dir_name}/hotdog/chester.png')","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:04.724266Z","iopub.execute_input":"2022-09-16T21:40:04.724794Z","iopub.status.idle":"2022-09-16T21:40:04.731823Z","shell.execute_reply.started":"2022-09-16T21:40:04.724741Z","shell.execute_reply":"2022-09-16T21:40:04.730389Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import urllib.request\nurllib.request.urlretrieve(\"https://d3mvlb3hz2g78.cloudfront.net/wp-content/uploads/2014/06/thumb_720_450_dreamstime_xl_34122178-Custom.jpg\", \"hotdog.jpg\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:04.733848Z","iopub.execute_input":"2022-09-16T21:40:04.735166Z","iopub.status.idle":"2022-09-16T21:40:06.971281Z","shell.execute_reply.started":"2022-09-16T21:40:04.735110Z","shell.execute_reply":"2022-09-16T21:40:06.969790Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\nIMAGE_DIMS = (224, 224)\n\n#img = Im.open(f\"{dir_name}/hotdog/chester.png\")\nimg = Im.open(\"/kaggle/working/hotdog.jpg\")\nimg = img.resize(IMAGE_DIMS)\nimg_bytes = img.tobytes()\nb64_img_bytes = base64.urlsafe_b64encode(img_bytes)\n# Always good to see it\nimg","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:06.974745Z","iopub.execute_input":"2022-09-16T21:40:06.975600Z","iopub.status.idle":"2022-09-16T21:40:07.020702Z","shell.execute_reply.started":"2022-09-16T21:40:06.975552Z","shell.execute_reply":"2022-09-16T21:40:07.019396Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"try:\n r = requests.post(\"https://hotdog.fly.dev/score\", data=json.dumps({\"input\": b64_img_bytes.decode()}))\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:07.022989Z","iopub.execute_input":"2022-09-16T21:40:07.023439Z","iopub.status.idle":"2022-09-16T21:40:07.400982Z","shell.execute_reply.started":"2022-09-16T21:40:07.023396Z","shell.execute_reply":"2022-09-16T21:40:07.399374Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"hotdog\", \"APOI0VROVXVM7G}TRXP5Q8CRFNZ6FGZI9JHOATCFTLWOFSYANZRHRCJPDLDURYRSO39Z21SIBZO9YTDDNK45FOVF9F0DDX8EXBHATCDLO6DMG1RRWU7Z5GYG3KS{DB0L\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:07.403811Z","iopub.execute_input":"2022-09-16T21:40:07.404660Z","iopub.status.idle":"2022-09-16T21:40:07.424810Z","shell.execute_reply.started":"2022-09-16T21:40:07.404614Z","shell.execute_reply":"2022-09-16T21:40:07.423612Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n# Math Challenge 1 (id: math_1) -- 100 Points\n\nFor challenge 1 I used the [the following code](https://www.geeksforgeeks.org/determining-the-number-of-clusters-in-data-mining/) to jump start me down the path of finding the number of clusters.","metadata":{}},{"cell_type":"code","source":"# Load everything into pandas\nb1 = np.load('/kaggle/input/ai-village-ctf/math/clusters1.npy')\nb2 = np.load('/kaggle/input/ai-village-ctf/math/clusters2.npy')\nb3 = np.load('/kaggle/input/ai-village-ctf/math/clusters3.npy')","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:07.426184Z","iopub.execute_input":"2022-09-16T21:40:07.427398Z","iopub.status.idle":"2022-09-16T21:40:07.463789Z","shell.execute_reply.started":"2022-09-16T21:40:07.427352Z","shell.execute_reply":"2022-09-16T21:40:07.462675Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom sklearn.cluster import KMeans\nfrom sklearn.metrics import silhouette_score\n\ndataset_new = b3\n\nlimit = int((dataset_new.shape[0]//2)**0.5)\n \n# determining number of clusters\n# using silhouette score method\nfor k in range(2, limit+1):\n model = KMeans(n_clusters=k)\n model.fit(dataset_new)\n pred = model.predict(dataset_new)\n score = silhouette_score(dataset_new, pred)\n print('Silhouette Score for k = {}: {:<.3f}'.format(k, score))\n\n #b1=5\n #b2=2\n #b3=3","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:07.465218Z","iopub.execute_input":"2022-09-16T21:40:07.466065Z","iopub.status.idle":"2022-09-16T21:40:07.932488Z","shell.execute_reply.started":"2022-09-16T21:40:07.466018Z","shell.execute_reply":"2022-09-16T21:40:07.931247Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n#2600,1875,555\ntry:\n r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"math_1\", \"submission\": \"523\"}))\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:07.934587Z","iopub.execute_input":"2022-09-16T21:40:07.935983Z","iopub.status.idle":"2022-09-16T21:40:08.095653Z","shell.execute_reply.started":"2022-09-16T21:40:07.935929Z","shell.execute_reply":"2022-09-16T21:40:08.094296Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"math_1\", \"BJYXGHKP8WH6ASH4882QR38U3TK{ZF8B2DW4Z1OTCDT8NSB43JIL10CGTTNGMIZY}EUGYVLL7I6PZ53I0DAV1AFGCMSQYIPQGHA7RR7TJJAOX7N12J7NLCIGNLFHIS0D\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:08.102245Z","iopub.execute_input":"2022-09-16T21:40:08.102932Z","iopub.status.idle":"2022-09-16T21:40:08.116038Z","shell.execute_reply.started":"2022-09-16T21:40:08.102882Z","shell.execute_reply":"2022-09-16T21:40:08.114778Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n# Math Challenge 2 (id: math_2) -- 200 Points\n\nI didn't really understand the question and what was being looked for. So I played around a bit but just bruteforced in the end.\n\n\n\n\n\n* https://www.kaggle.com/competitions/ai-village-ctf/discussion/344190\n* https://towardsdatascience.com/principal-component-analysis-for-dimensionality-reduction-115a3d157bad","metadata":{}},{"cell_type":"code","source":"# Correct answer from https://www.kaggle.com/code/iraqai/d3fc0n30-ctf-writeup-with-logic-no-luck\nimport numpy as np\nfrom sklearn.decomposition import PCA\ndef get_mydim(f):\n X = np.load(f)\n pca = PCA(n_components='mle')\n d_out = pca.fit_transform(X)\n return d_out","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:08.117917Z","iopub.execute_input":"2022-09-16T21:40:08.118634Z","iopub.status.idle":"2022-09-16T21:40:08.125893Z","shell.execute_reply.started":"2022-09-16T21:40:08.118591Z","shell.execute_reply":"2022-09-16T21:40:08.124717Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(get_mydim('../input/ai-village-ctf/math/first_dim1.npy').shape)\nprint(get_mydim('../input/ai-village-ctf/math/first_dim2.npy').shape)\nprint(get_mydim('../input/ai-village-ctf/math/first_dim3.npy').shape)","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:08.127692Z","iopub.execute_input":"2022-09-16T21:40:08.128445Z","iopub.status.idle":"2022-09-16T21:40:08.172856Z","shell.execute_reply.started":"2022-09-16T21:40:08.128400Z","shell.execute_reply":"2022-09-16T21:40:08.171584Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls /kaggle/input/ai-village-ctf/math","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:08.174877Z","iopub.execute_input":"2022-09-16T21:40:08.175658Z","iopub.status.idle":"2022-09-16T21:40:09.239056Z","shell.execute_reply.started":"2022-09-16T21:40:08.175613Z","shell.execute_reply":"2022-09-16T21:40:09.237856Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"d1 = np.load('/kaggle/input/ai-village-ctf/math/first_dim1.npy')\nd2 = np.load('/kaggle/input/ai-village-ctf/math/first_dim2.npy')\nd3 = np.load('/kaggle/input/ai-village-ctf/math/first_dim3.npy')","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:09.240774Z","iopub.execute_input":"2022-09-16T21:40:09.241847Z","iopub.status.idle":"2022-09-16T21:40:09.250726Z","shell.execute_reply.started":"2022-09-16T21:40:09.241805Z","shell.execute_reply":"2022-09-16T21:40:09.249500Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(d1.ndim)","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:09.252230Z","iopub.execute_input":"2022-09-16T21:40:09.252612Z","iopub.status.idle":"2022-09-16T21:40:09.259485Z","shell.execute_reply.started":"2022-09-16T21:40:09.252580Z","shell.execute_reply":"2022-09-16T21:40:09.258385Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"d1.shape, d1.size, d1.ndim\n#100","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:09.261111Z","iopub.execute_input":"2022-09-16T21:40:09.261480Z","iopub.status.idle":"2022-09-16T21:40:09.273225Z","shell.execute_reply.started":"2022-09-16T21:40:09.261439Z","shell.execute_reply":"2022-09-16T21:40:09.272156Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\n\ndf = pd.DataFrame(d1)\n\nprint(df)\nprint(type(df))","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:09.274837Z","iopub.execute_input":"2022-09-16T21:40:09.275190Z","iopub.status.idle":"2022-09-16T21:40:09.289913Z","shell.execute_reply.started":"2022-09-16T21:40:09.275152Z","shell.execute_reply":"2022-09-16T21:40:09.288760Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"d2.shape, d2.size, d2.ndim","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:09.291726Z","iopub.execute_input":"2022-09-16T21:40:09.292071Z","iopub.status.idle":"2022-09-16T21:40:09.303434Z","shell.execute_reply.started":"2022-09-16T21:40:09.292041Z","shell.execute_reply":"2022-09-16T21:40:09.302299Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"d3.shape, d3.size, d3.ndim\n#120","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:09.305026Z","iopub.execute_input":"2022-09-16T21:40:09.305390Z","iopub.status.idle":"2022-09-16T21:40:09.313126Z","shell.execute_reply.started":"2022-09-16T21:40:09.305359Z","shell.execute_reply":"2022-09-16T21:40:09.312256Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"try:\n answer = \"354\"\n r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"math_2\", \"submission\": answer}))\n print(answer+\" \"+r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:09.314083Z","iopub.execute_input":"2022-09-16T21:40:09.314460Z","iopub.status.idle":"2022-09-16T21:40:09.450349Z","shell.execute_reply.started":"2022-09-16T21:40:09.314428Z","shell.execute_reply":"2022-09-16T21:40:09.449060Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"math_2\", \"SCHXNVG301DALRULS8EUFQXL3L}ERIJV0RBV881XUZTXHOXFO5V8ZBBYA4OMACQ1AO6P23R6JSFCJPA2K6LAIZICTAMHHGT4AGDTKFMMRPUXMW2ZI{77GPQKADWCTRMH\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:09.452537Z","iopub.execute_input":"2022-09-16T21:40:09.453490Z","iopub.status.idle":"2022-09-16T21:40:09.466884Z","shell.execute_reply.started":"2022-09-16T21:40:09.453444Z","shell.execute_reply":"2022-09-16T21:40:09.465293Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Math Challenge 3 (id: math_3) -- 300 Points\n\nI bruteforced 3 and 4 since code for 2 was already done. My favorite solution was [here](https://www.kaggle.com/code/vkonstantakos/math-challenges-dbscan-pca?scriptVersionId=105441231)","metadata":{}},{"cell_type":"code","source":"d1 = np.load('/kaggle/input/ai-village-ctf/math/second_dim1.npy')\nd2 = np.load('/kaggle/input/ai-village-ctf/math/second_dim2.npy')\nd3 = np.load('/kaggle/input/ai-village-ctf/math/second_dim3.npy')","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:09.468509Z","iopub.execute_input":"2022-09-16T21:40:09.469411Z","iopub.status.idle":"2022-09-16T21:40:09.489631Z","shell.execute_reply.started":"2022-09-16T21:40:09.469365Z","shell.execute_reply":"2022-09-16T21:40:09.488765Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\ntry:\n r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"math_3\", \"submission\": \"474\"}))\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:09.490989Z","iopub.execute_input":"2022-09-16T21:40:09.491881Z","iopub.status.idle":"2022-09-16T21:40:09.634166Z","shell.execute_reply.started":"2022-09-16T21:40:09.491826Z","shell.execute_reply":"2022-09-16T21:40:09.632899Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"math_3\", \"5Y5ZL4F3BHVYXQHGFZB0OX2OS}{8V51JJTMT91R3VGLQCW044AGHPN796WLGXJJ4ZM44WEUX9YU_25S1DWQ9AAWIQYZ4CXGAESX2F7ANWZONVG56FPQPFT8L6RTMXXHR\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:09.635757Z","iopub.execute_input":"2022-09-16T21:40:09.636925Z","iopub.status.idle":"2022-09-16T21:40:09.649061Z","shell.execute_reply.started":"2022-09-16T21:40:09.636877Z","shell.execute_reply":"2022-09-16T21:40:09.647629Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Math Challenge 4 (id: math_4) -- 400 Points","metadata":{}},{"cell_type":"code","source":"d1 = np.load('/kaggle/input/ai-village-ctf/math/clusters1.npy')","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:09.651217Z","iopub.execute_input":"2022-09-16T21:40:09.652090Z","iopub.status.idle":"2022-09-16T21:40:09.658592Z","shell.execute_reply.started":"2022-09-16T21:40:09.652044Z","shell.execute_reply":"2022-09-16T21:40:09.657367Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#54312\ntry:\n answer = \"54312\"\n r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"math_4\", \"submission\": answer}))\n print(answer+\" \"+r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:09.661073Z","iopub.execute_input":"2022-09-16T21:40:09.661836Z","iopub.status.idle":"2022-09-16T21:40:09.809807Z","shell.execute_reply.started":"2022-09-16T21:40:09.661791Z","shell.execute_reply":"2022-09-16T21:40:09.808462Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"math_4\", \"2SXZCVGSK9NF0EOB7W5HDF9G0AC1LHWBHLT}1SBIVZTY5PS9PNNTPEIH1Q7WUWPFRFI590RZIAB51RTHN4JJ662MPX74PRV3F5X{MFGXAPSQPJZ9KE4IME6GAK6OVVG0\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:09.811812Z","iopub.execute_input":"2022-09-16T21:40:09.812286Z","iopub.status.idle":"2022-09-16T21:40:09.824798Z","shell.execute_reply.started":"2022-09-16T21:40:09.812242Z","shell.execute_reply":"2022-09-16T21:40:09.823395Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n# Honorstudent (id: honorstudent) -- 300 Points\n\nThis one was a quick photoshop away of drawing an A. I feel mine was a little heavy handed. I am an artist!\n\n\n\n\n\n\n\n","metadata":{}},{"cell_type":"code","source":"Ima(filename=f'{dir_name}/honorstudent/f.png')","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:09.836004Z","iopub.execute_input":"2022-09-16T21:40:09.836789Z","iopub.status.idle":"2022-09-16T21:40:09.860505Z","shell.execute_reply.started":"2022-09-16T21:40:09.836739Z","shell.execute_reply":"2022-09-16T21:40:09.859208Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import urllib.request\n\nurllib.request.urlretrieve(\"https://i.ibb.co/L9mbcYT/A2.png\", \"A.png\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:09.862151Z","iopub.execute_input":"2022-09-16T21:40:09.863412Z","iopub.status.idle":"2022-09-16T21:40:10.334930Z","shell.execute_reply.started":"2022-09-16T21:40:09.863364Z","shell.execute_reply":"2022-09-16T21:40:10.333643Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper- Look at that little A there!\n\nimg = Im.open(\"/kaggle/working/A.png\")\nimg","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:10.336547Z","iopub.execute_input":"2022-09-16T21:40:10.337210Z","iopub.status.idle":"2022-09-16T21:40:10.347692Z","shell.execute_reply.started":"2022-09-16T21:40:10.337164Z","shell.execute_reply":"2022-09-16T21:40:10.346475Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"with open(\"/kaggle/working/A.png\", \"rb\") as f:\n try:\n r = requests.post(\"https://honorstudent.fly.dev/score\", files={\"data_file\": f})\n print(r.text)\n except requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:10.349524Z","iopub.execute_input":"2022-09-16T21:40:10.350159Z","iopub.status.idle":"2022-09-16T21:40:10.525069Z","shell.execute_reply.started":"2022-09-16T21:40:10.350115Z","shell.execute_reply":"2022-09-16T21:40:10.523694Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"honorstudent\", \"TYSHK4A6MGYSGMZGAJKSLK5MC5BVIGU48M0UA8KSN1F7}UG123RO401GA8OPNVARTWFL7LBRUZUEUVSF59JTEKP9VS{YOALCH945CNC7FDPHT2J2NG6CQT35K7PEP6LE\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:10.527458Z","iopub.execute_input":"2022-09-16T21:40:10.528792Z","iopub.status.idle":"2022-09-16T21:40:10.541276Z","shell.execute_reply.started":"2022-09-16T21:40:10.528743Z","shell.execute_reply":"2022-09-16T21:40:10.540145Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n# Bad to Good (id: bad_to_good) -- 300 Points\nThis one I spent too much time trying to play around just with a few groups. I thought I could not change the Henry's grade (Which I could..). Thinking it was somehow related to grouping the student with the \"good\" kids together until I moved Henry's grade better as well.\n\n","metadata":{}},{"cell_type":"code","source":"!ls /kaggle/input/ai-village-ctf/bad2good/","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:10.543347Z","iopub.execute_input":"2022-09-16T21:40:10.544141Z","iopub.status.idle":"2022-09-16T21:40:11.599142Z","shell.execute_reply.started":"2022-09-16T21:40:10.544097Z","shell.execute_reply":"2022-09-16T21:40:11.597570Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pandas as pd\n\ndf = pd.read_csv(\"/kaggle/input/ai-village-ctf/bad2good/math.csv\")\nprint (df)","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:11.601923Z","iopub.execute_input":"2022-09-16T21:40:11.602815Z","iopub.status.idle":"2022-09-16T21:40:11.622915Z","shell.execute_reply.started":"2022-09-16T21:40:11.602763Z","shell.execute_reply":"2022-09-16T21:40:11.621642Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# What are the best scores and grades\nlscore=df['Score'].nlargest(n=30)\nlgrade=df['Grade'].nlargest(n=30)","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:11.625065Z","iopub.execute_input":"2022-09-16T21:40:11.625961Z","iopub.status.idle":"2022-09-16T21:40:11.635115Z","shell.execute_reply.started":"2022-09-16T21:40:11.625911Z","shell.execute_reply":"2022-09-16T21:40:11.633914Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#hHat are the X best grades... lets make them as good as Henry. Everyone else gets some demerits...\ntop=12\nfor i in lscore.index[0:top]:\n #print (i)\n df.loc[i, 'Demerits'] = 0\n df.loc[i, 'Absences'] = 0\n df.loc[i, 'Score'] = 57\n #df.loc[i, 'Grade'] = 67\n \nfor i in lscore.index[(top+1):30]:\n #print (i)\n df.loc[i, 'Demerits'] = 5\n df.loc[i, 'Absences'] = 3\n df.loc[i, 'Score'] = 15\n #df.loc[i, 'Grade'] = 67","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:11.636713Z","iopub.execute_input":"2022-09-16T21:40:11.637510Z","iopub.status.idle":"2022-09-16T21:40:11.667406Z","shell.execute_reply.started":"2022-09-16T21:40:11.637466Z","shell.execute_reply":"2022-09-16T21:40:11.666168Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\n#Runs the equation multiple times.\n\ndef manytimes(top1,demerits,grade):\n df1= pd.read_csv(\"/kaggle/input/ai-village-ctf/bad2good/math.csv\")\n lscore=df1['Score'].nlargest(n=30)\n lgrade=df1['Grade'].nlargest(n=30)\n top = top1\n #top students like me\n for i in lscore.index[0:top]:\n #print (i)\n df1.loc[i, 'Demerits'] = 0\n df1.loc[i, 'Absences'] = 0\n df1.loc[i, 'Score'] = 15\n #df.loc[i, 'Grade'] = 67\n for i in lscore.index[(top+1):30]:\n #print (i)\n df1.loc[i, 'Demerits'] = demerits\n df1.loc[i, 'Absences'] = demerits\n df1.loc[i, 'Score'] = 15\n #df.loc[i, 'Grade'] = 67\n # Make sure Hacker doesntchange\n df1.loc[df1.StudentID == 1337, 'Demerits'] = 0\n df1.loc[df1.StudentID == 1337, 'Absences'] = 0\n df1.loc[df1.StudentID == 1337, 'Grade'] = grade\n df1.loc[df1.StudentID == 1337, 'Score'] = 100\n\n df1.to_csv(\"/kaggle/working/math1.csv\")\n print (df1)\n# Submission Helper\n with open(\"/kaggle/working/math1.csv\", \"r\") as f:\n try:\n r = requests.post(\"https://bad2good.fly.dev/score\", files={\"data_file\": f})\n print(\"The \"+str(top)+\" Demerits\"+ str(demerits)+r.text)\n except requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:11.669362Z","iopub.execute_input":"2022-09-16T21:40:11.670377Z","iopub.status.idle":"2022-09-16T21:40:11.684100Z","shell.execute_reply.started":"2022-09-16T21:40:11.670301Z","shell.execute_reply":"2022-09-16T21:40:11.682755Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Oh I can change grades... guess I will \nfor x in range(24, 25): #top number of students Higher the better\n for y in range(0, 1): #demerits\n for grade in range(190,191):\n manytimes(x,y,grade)","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:11.686005Z","iopub.execute_input":"2022-09-16T21:40:11.686494Z","iopub.status.idle":"2022-09-16T21:40:11.909293Z","shell.execute_reply.started":"2022-09-16T21:40:11.686448Z","shell.execute_reply":"2022-09-16T21:40:11.907918Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Final Talley\ndf","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:11.911771Z","iopub.execute_input":"2022-09-16T21:40:11.912770Z","iopub.status.idle":"2022-09-16T21:40:11.935814Z","shell.execute_reply.started":"2022-09-16T21:40:11.912720Z","shell.execute_reply":"2022-09-16T21:40:11.934394Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"bad_to_good\", \"wtHpXFJBh6UZNUG3nAXF0226HSMHIKJTFKH2B01HX_S77RGUQRWOGN0tW4MGW6A_AZMO97CeYBLDM9aVFL2K}bRLIHYBV6NGo9C6I7F9VSW8SQ_TH3CVURd3S{C4KZHQ\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:11.937604Z","iopub.execute_input":"2022-09-16T21:40:11.938409Z","iopub.status.idle":"2022-09-16T21:40:11.950389Z","shell.execute_reply.started":"2022-09-16T21:40:11.938362Z","shell.execute_reply":"2022-09-16T21:40:11.949135Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n# Baseball (id: baseball) -- 300 Points\n\nThis one was a mind freak for me, but I did make some pretty graphs. At first I believed that it mattered where Henry through the ball and spent a day gathering information of what was most likely. \n\nFirst I tried finding high \"Henry\" scores, took all coorinates that were in successful \"Henry\" scores to build a super list of coordinates that showed up the most for \"Henry\"... except it turned into a Sarah....\n\nLater I went back captured the highest score I could from brute forcing, and then ran it through some backprop to find a better fit. The problem was I thought the output would say \"Henry\" and still have a confidence score above 70... it did not. It stalled around .699903. Which makes sense because my code just dropped the flag response. So I spent extra days playing around trying to understand everything better.\n\n\n\nBy putting it in a tabular form I was able to go back and start building out additional features from large sets of data. I started to see that it wasn't based on the X,Y but other features. When I went back to get some high confidence data with George and Joe I rewrote the errors and got the flag.\n\nLesson learned","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nfrom numpy.random import randint\nfrom random import randrange\nimport matplotlib.pyplot as plt\nimport seaborn as sns","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:11.952141Z","iopub.execute_input":"2022-09-16T21:40:11.952862Z","iopub.status.idle":"2022-09-16T21:40:11.961726Z","shell.execute_reply.started":"2022-09-16T21:40:11.952818Z","shell.execute_reply":"2022-09-16T21:40:11.960835Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def mapshow(dflarge):\n # Control figure size for this notebook:\n plt.rcParams['figure.figsize'] = [8, 8]\n sns.lmplot('x', 'y', data=dflarge, fit_reg=False)\n plt.xlabel('x')\n # Set y-axis label\n plt.ylabel('y')\n #sns.lmplot(data=[2,2], fit_reg=False)\n plt.scatter(sum(dflarge['x'])/len(dflarge), sum(dflarge['y'])/len(dflarge), marker='o', s=100)\n plt.show()\n\ndef baseball(x,y,test1):\n try:\n r = requests.post(\"https://bball.fly.dev/score\", json=test1)\n return(\"x \"+str(x)+\" y \"+str(y)+\" \"+r.text)\n except requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")\n \ndef baseball2(test1):\n try:\n r = requests.post(\"https://bball.fly.dev/score\", json=test1)\n return(r.text)\n except requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:11.963349Z","iopub.execute_input":"2022-09-16T21:40:11.964071Z","iopub.status.idle":"2022-09-16T21:40:11.976422Z","shell.execute_reply.started":"2022-09-16T21:40:11.964025Z","shell.execute_reply":"2022-09-16T21:40:11.975269Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!cat /kaggle/input/ai-village-ctf/baseball/test_pitch.json","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:11.980523Z","iopub.execute_input":"2022-09-16T21:40:11.981331Z","iopub.status.idle":"2022-09-16T21:40:13.020943Z","shell.execute_reply.started":"2022-09-16T21:40:11.981263Z","shell.execute_reply":"2022-09-16T21:40:13.019706Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"with open(f\"{dir_name}/baseball/test_pitch.json\", \"r\") as f:\n test = json.load(f)\n\n#Build a random set of pitches\nfor t in range(0, 15):\n test[\"pitches\"][t][\"x\"]=randrange(29)\n test[\"pitches\"][t][\"y\"]=randrange(29)\nbaseball2(test)","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:13.023328Z","iopub.execute_input":"2022-09-16T21:40:13.023845Z","iopub.status.idle":"2022-09-16T21:40:13.390573Z","shell.execute_reply.started":"2022-09-16T21:40:13.023792Z","shell.execute_reply":"2022-09-16T21:40:13.389351Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dfplots = {'x':[],\n 'y':[],\n 'conf':[],\n 'count':[]\n }\n\ndfplots = pd.DataFrame(dfplots)\n\n#build plot list\nfor i in range(0,29):\n for i2 in range(1,29):\n dfplots.loc[len(dfplots.index)] = [i, i2, 0,0] ","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:13.392427Z","iopub.execute_input":"2022-09-16T21:40:13.393159Z","iopub.status.idle":"2022-09-16T21:40:14.601460Z","shell.execute_reply.started":"2022-09-16T21:40:13.393114Z","shell.execute_reply":"2022-09-16T21:40:14.600431Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#If Henry make bigger, otherwise make smaller\ndef scoring(dfplots, test, score):\n for t in range(0, 15):\n xx=test[\"pitches\"][t][\"x\"]\n xy=test[\"pitches\"][t][\"y\"]\n c = dfplots['x'].eq(xx) & dfplots['y'].eq(xy)\n dfplots.loc[c,'conf']=dfplots.loc[c,'conf']+score\n dfplots.loc[c,'count']=dfplots.loc[c,'count']+1\n return dfplots","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:14.603063Z","iopub.execute_input":"2022-09-16T21:40:14.603806Z","iopub.status.idle":"2022-09-16T21:40:14.612640Z","shell.execute_reply.started":"2022-09-16T21:40:14.603759Z","shell.execute_reply":"2022-09-16T21:40:14.611345Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#dfplots=scoring(dfplots, test, 1)\ndfplots","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:14.614261Z","iopub.execute_input":"2022-09-16T21:40:14.615003Z","iopub.status.idle":"2022-09-16T21:40:14.643564Z","shell.execute_reply.started":"2022-09-16T21:40:14.614954Z","shell.execute_reply":"2022-09-16T21:40:14.641855Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"### joe, george, sarah, henry\n# This makes random ptiches and then checks then against who they matched with to build a panda\n\ndef generateData(dfa, runs):\n for i in range(0,runs):#15000):\n #Make a random sample\n for t in range(0, 15):\n test[\"pitches\"][t][\"x\"]=randrange(29)\n test[\"pitches\"][t][\"y\"]=randrange(29)\n\n #Split the result\n check=baseball2(test)\n\n if \"henry\" in check:\n check1=check.split(\"Confidence: \")\n check2=check1[1].split('.\", 200]}')\n score=float(check2[0])\n #dfplots=scoring(dfplots, test, score)\n name=\"henry\"\n elif \"sarah\" in check:\n check1=check.split(\"liklihood of \")\n check2=check1[1].split('\", 200]}')\n score=float(check2[0])*1\n #dfplots=scoring(dfplots, test, score)\n name=\"sarah\"\n elif \"joe\" in check:\n check1=check.split(\"liklihood of \")\n check2=check1[1].split('\", 200]}')\n score=float(check2[0])*1\n #dfplots=scoring(dfplots, test, score)\n name=\"joe\"\n elif \"george\" in check:\n check1=check.split(\"liklihood of \")\n check2=check1[1].split('\", 200]}')\n score=float(check2[0])*1\n #dfplots=scoring(dfplots, test, score)\n name=\"george\"\n elif \"Not 15 unique pitches.\" in check:\n continue\n else:\n print (\"Something went wrong \"+str(check))\n continue\n\n dfa.loc[len(dfa.index)] = [test['pitches'][0]['x'], test['pitches'][0]['y'],\n test['pitches'][1]['x'], test['pitches'][1]['y'],\n test['pitches'][2]['x'], test['pitches'][2]['y'],\n test['pitches'][3]['x'], test['pitches'][3]['y'],\n test['pitches'][4]['x'], test['pitches'][4]['y'],\n test['pitches'][5]['x'], test['pitches'][5]['y'],\n test['pitches'][6]['x'], test['pitches'][6]['y'],\n test['pitches'][7]['x'], test['pitches'][7]['y'],\n test['pitches'][8]['x'], test['pitches'][8]['y'],\n test['pitches'][9]['x'], test['pitches'][9]['y'],\n test['pitches'][10]['x'], test['pitches'][10]['y'],\n test['pitches'][11]['x'], test['pitches'][11]['y'],\n test['pitches'][12]['x'], test['pitches'][12]['y'],\n test['pitches'][13]['x'], test['pitches'][13]['y'],\n test['pitches'][14]['x'], test['pitches'][14]['y'],\n '0','0',float(check2[0]),name]\n return dfa","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:14.645485Z","iopub.execute_input":"2022-09-16T21:40:14.645984Z","iopub.status.idle":"2022-09-16T21:40:14.669659Z","shell.execute_reply.started":"2022-09-16T21:40:14.645936Z","shell.execute_reply":"2022-09-16T21:40:14.668179Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# We made some pitches lets see if they still work\ntry:\n dfa = pickle.load(open('../input/small-dataset/82000.sav','rb'))\n pickle.dump(dfa, open(\"./dfa.sav\", 'wb'))\nexcept:\n print (\"No Input Data\")\ntry:\n dfa = pickle.load(open('./dfa.sav','rb'))\nexcept:\n dfa = pd.DataFrame(columns=['x1','y1',\n 'x2','y2',\n 'x3','y3',\n 'x4','y4',\n 'x5','y5',\n 'x6','y6',\n 'x7','y7',\n 'x8','y8',\n 'x9','y9',\n 'x10','y10',\n 'x11','y11',\n 'x12','y12',\n 'x13','y13',\n 'x14','y14',\n 'x15','y15',\n 'midx','midy','conf','name'])\n generateData(dfa, 200)\n pickle.dump(dfa, open(\"./dfa.sav\", 'wb'))\n","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:14.671483Z","iopub.execute_input":"2022-09-16T21:40:14.671955Z","iopub.status.idle":"2022-09-16T21:40:14.745395Z","shell.execute_reply.started":"2022-09-16T21:40:14.671907Z","shell.execute_reply":"2022-09-16T21:40:14.744116Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#you can make more datapoints\ndfa=generateData(dfa, 1)\nprint (\"You now have \"+str(len(dfa.index))+\" datapoints\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:14.749419Z","iopub.execute_input":"2022-09-16T21:40:14.749844Z","iopub.status.idle":"2022-09-16T21:40:15.091927Z","shell.execute_reply.started":"2022-09-16T21:40:14.749808Z","shell.execute_reply":"2022-09-16T21:40:15.090746Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pickle.dump(dfa, open(\"./dfa20000.sav\", 'wb'))","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:15.093379Z","iopub.execute_input":"2022-09-16T21:40:15.093771Z","iopub.status.idle":"2022-09-16T21:40:15.114499Z","shell.execute_reply.started":"2022-09-16T21:40:15.093738Z","shell.execute_reply":"2022-09-16T21:40:15.113277Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#For awhile I was making a list of pitches that equated back\nhenrylist=[]\nsarahlist=[]\ngeorgelist=[]\njoelist=[]","metadata":{"execution":{"iopub.status.busy":"2022-09-12T03:14:16.122512Z","iopub.execute_input":"2022-09-12T03:14:16.123805Z","iopub.status.idle":"2022-09-12T03:14:16.129571Z","shell.execute_reply.started":"2022-09-12T03:14:16.123755Z","shell.execute_reply":"2022-09-12T03:14:16.128528Z"}}},{"cell_type":"code","source":"# Lets add some features!\n\ndfa['midx']=(dfa['x1']+dfa['x2']+dfa['x3']+dfa['x4']+dfa['x5']+dfa['x6']+dfa['x7']+dfa['x8']+dfa['x9']+dfa['x10']+dfa['x11']+dfa['x12']+dfa['x13']+dfa['x14']+dfa['x15'])/15\ndfa['midy']=(dfa['y1']+dfa['y2']+dfa['y3']+dfa['y4']+dfa['y5']+dfa['y6']+dfa['y7']+dfa['y8']+dfa['y9']+dfa['y10']+dfa['y11']+dfa['y12']+dfa['y13']+dfa['y14']+dfa['y15'])/15\n\ndfa['maxX']=dfa[[\"x1\", \"x2\",\"x3\", \"x4\",\"x5\", \"x6\",\"x7\", \"x8\",\"x9\", \"x10\",\"x11\", \"x12\",\"x13\", \"x14\",\"x15\"]].max(axis=1)\ndfa['maxY']=dfa[[\"y1\", \"y2\",\"y3\", \"y4\",\"y5\", \"y6\",\"y7\", \"y8\",\"y9\", \"y10\",\"y11\", \"y12\",\"y13\", \"y14\",\"y15\"]].max(axis=1)\n\ndfa['minX']=dfa[[\"x1\", \"x2\",\"x3\", \"x4\",\"x5\", \"x6\",\"x7\", \"x8\",\"x9\", \"x10\",\"x11\", \"x12\",\"x13\", \"x14\",\"x15\"]].min(axis=1)\ndfa['minY']=dfa[[\"y1\", \"y2\",\"y3\", \"y4\",\"y5\", \"y6\",\"y7\", \"y8\",\"y9\", \"y10\",\"y11\", \"y12\",\"y13\", \"y14\",\"y15\"]].min(axis=1)\n\ndfa['lenX']=dfa['maxX']-dfa['minX']\ndfa['lenY']=dfa['maxY']-dfa['minY']\n\n#dfa['maxX']=max(dfa['x1'],dfa['x2'],dfa['x3'],dfa['x4'],dfa['x5'],dfa['x6'],dfa['x7'],dfa['x8'],dfa['x9'],dfa['x10'],dfa['x11'],dfa['x12'],dfa['x13'],dfa['x14'],dfa['x15'])","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:15.116806Z","iopub.execute_input":"2022-09-16T21:40:15.117439Z","iopub.status.idle":"2022-09-16T21:40:15.241347Z","shell.execute_reply.started":"2022-09-16T21:40:15.117393Z","shell.execute_reply":"2022-09-16T21:40:15.240196Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"listx=[\"x1\", \"x2\",\"x3\", \"x4\",\"x5\", \"x6\",\"x7\", \"x8\",\"x9\", \"x10\",\"x11\", \"x12\",\"x13\", \"x14\",\"x15\"]\nfor x in listx:\n dfa[x+\"midDist\"]=abs(dfa[x]-dfa['midx'])\n \nlisty=[\"y1\", \"y2\",\"y3\", \"y4\",\"y5\", \"y6\",\"y7\", \"y8\",\"y9\", \"y10\",\"y11\", \"y12\",\"y13\", \"y14\",\"y15\"]\nfor x in listy:\n dfa[x+\"midDist\"]=abs(dfa[x]-dfa['midy'])\n \ndfa['avgmidDistX']=(dfa['x1midDist']+dfa['x2midDist']+dfa['x3midDist']+dfa['x4midDist']+dfa['x5midDist']+dfa['x6midDist']+dfa['x7midDist']+dfa['x8midDist']+dfa['x9midDist']+dfa['x10midDist']+dfa['x11midDist']+dfa['x12midDist']+dfa['x13midDist']+dfa['x14midDist']+dfa['x15midDist'])/15\ndfa['avgmidDistY']=(dfa['y1midDist']+dfa['y2midDist']+dfa['y3midDist']+dfa['y4midDist']+dfa['y5midDist']+dfa['y6midDist']+dfa['y7midDist']+dfa['y8midDist']+dfa['y9midDist']+dfa['y10midDist']+dfa['y11midDist']+dfa['y12midDist']+dfa['y13midDist']+dfa['y14midDist']+dfa['y15midDist'])/15","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:15.242548Z","iopub.execute_input":"2022-09-16T21:40:15.243508Z","iopub.status.idle":"2022-09-16T21:40:15.354366Z","shell.execute_reply.started":"2022-09-16T21:40:15.243475Z","shell.execute_reply":"2022-09-16T21:40:15.353084Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"listx=[\"x1\", \"x2\",\"x3\", \"x4\",\"x5\", \"x6\",\"x7\", \"x8\",\"x9\", \"x10\",\"x11\", \"x12\",\"x13\", \"x14\",\"x15\"]\nfor x in listx:\n dfa[x+\"midDistCenter\"]=abs(dfa[x]-15)\n \nlisty=[\"y1\", \"y2\",\"y3\", \"y4\",\"y5\", \"y6\",\"y7\", \"y8\",\"y9\", \"y10\",\"y11\", \"y12\",\"y13\", \"y14\",\"y15\"]\nfor x in listy:\n dfa[x+\"midDistCenter\"]=abs(dfa[x]-15)\n \ndfa['avgmidDistXCenter']=(dfa['x1midDist']+dfa['x2midDist']+dfa['x3midDist']+dfa['x4midDist']+dfa['x5midDist']+dfa['x6midDist']+dfa['x7midDist']+dfa['x8midDist']+dfa['x9midDist']+dfa['x10midDist']+dfa['x11midDist']+dfa['x12midDist']+dfa['x13midDist']+dfa['x14midDist']+dfa['x15midDist'])/15\ndfa['avgmidDistYCenter']=(dfa['y1midDist']+dfa['y2midDist']+dfa['y3midDist']+dfa['y4midDist']+dfa['y5midDist']+dfa['y6midDist']+dfa['y7midDist']+dfa['y8midDist']+dfa['y9midDist']+dfa['y10midDist']+dfa['y11midDist']+dfa['y12midDist']+dfa['y13midDist']+dfa['y14midDist']+dfa['y15midDist'])/15","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:15.355977Z","iopub.execute_input":"2022-09-16T21:40:15.356455Z","iopub.status.idle":"2022-09-16T21:40:15.426576Z","shell.execute_reply.started":"2022-09-16T21:40:15.356409Z","shell.execute_reply":"2022-09-16T21:40:15.425257Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dfa=dfa.infer_objects()\ndfa.info()","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:15.428261Z","iopub.execute_input":"2022-09-16T21:40:15.428735Z","iopub.status.idle":"2022-09-16T21:40:15.625295Z","shell.execute_reply.started":"2022-09-16T21:40:15.428688Z","shell.execute_reply":"2022-09-16T21:40:15.623987Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dfa.describe()","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:15.627030Z","iopub.execute_input":"2022-09-16T21:40:15.627703Z","iopub.status.idle":"2022-09-16T21:40:15.921140Z","shell.execute_reply.started":"2022-09-16T21:40:15.627641Z","shell.execute_reply":"2022-09-16T21:40:15.919451Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dfhenry = dfa[dfa['name'] == 'henry']\ndfsarah = dfa[dfa['name'] == 'sarah']\ndfjoe = dfa[dfa['name'] == 'joe']\ndfgeorge = dfa[dfa['name'] == 'george']","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:15.923227Z","iopub.execute_input":"2022-09-16T21:40:15.923728Z","iopub.status.idle":"2022-09-16T21:40:15.948185Z","shell.execute_reply.started":"2022-09-16T21:40:15.923682Z","shell.execute_reply":"2022-09-16T21:40:15.944337Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import matplotlib.pyplot as plt\nimport seaborn as sns\n\n# Control figure size for this notebook:\nplt.rcParams['figure.figsize'] = [8, 8]\n\n# data \ndata = test\n\nxs='lenX'\nys='lenY'\n \n# use the scatterplot function to build the bubble map\nsns.scatterplot(data=dfhenry, x=xs, y=ys, legend=True, sizes=(20, 2000))\nsns.scatterplot(data=dfsarah[:300], x=xs, y=ys, legend=True, sizes=(20, 2000))\nsns.scatterplot(data=dfjoe, x=xs, y=ys, legend=True, sizes=(20, 2000))\n#sns.scatterplot(data=dfgeorge, x=xs, y=ys, legend=True, sizes=(20, 2000))\n\n# show the graph\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:15.949648Z","iopub.execute_input":"2022-09-16T21:40:15.949974Z","iopub.status.idle":"2022-09-16T21:40:16.268779Z","shell.execute_reply.started":"2022-09-16T21:40:15.949944Z","shell.execute_reply":"2022-09-16T21:40:16.267337Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\n# Control figure size for this notebook:\nplt.rcParams['figure.figsize'] = [8, 8]\n\n# data \ndata = test\n\nxs='avgmidDistX'\nys='avgmidDistY'\n \n# use the scatterplot function to build the bubble map\nsns.scatterplot(data=dfhenry, x=xs, y=ys, legend=True, sizes=(20, 2000))\nsns.scatterplot(data=dfsarah[:300], x=xs, y=ys, legend=True, sizes=(20, 2000))\nsns.scatterplot(data=dfjoe, x=xs, y=ys, legend=True, sizes=(20, 2000))\n#sns.scatterplot(data=dfgeorge, x=xs, y=ys, legend=True, sizes=(20, 2000))\n\n# show the graph\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:16.270418Z","iopub.execute_input":"2022-09-16T21:40:16.270892Z","iopub.status.idle":"2022-09-16T21:40:16.530441Z","shell.execute_reply.started":"2022-09-16T21:40:16.270856Z","shell.execute_reply":"2022-09-16T21:40:16.529368Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\n# Control figure size for this notebook:\nplt.rcParams['figure.figsize'] = [8, 8]\n\n# data \ndata = test\n\nxs='avgmidDistX'\nys='avgmidDistY'\n \n# use the scatterplot function to build the bubble map\nsns.scatterplot(data=dfhenry, x=xs, y=ys, legend=True, sizes=(20, 2000))\nsns.scatterplot(data=dfsarah[:300], x=xs, y=ys, legend=True, sizes=(20, 2000))\nsns.scatterplot(data=dfjoe, x=xs, y=ys, legend=True, sizes=(20, 2000))\n#sns.scatterplot(data=dfgeorge, x=xs, y=ys, legend=True, sizes=(20, 2000))\n\n# show the graph\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:16.531874Z","iopub.execute_input":"2022-09-16T21:40:16.532920Z","iopub.status.idle":"2022-09-16T21:40:16.791178Z","shell.execute_reply.started":"2022-09-16T21:40:16.532883Z","shell.execute_reply":"2022-09-16T21:40:16.790047Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"### In Markdown because of error when empty\n\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\n# Control figure size for this notebook:\nplt.rcParams['figure.figsize'] = [8, 8]\n\n# data \ndata = test\n\nxs='avgmidDistXCenter'\nys='avgmidDistYCenter'\n \n# use the scatterplot function to build the bubble map\nsns.scatterplot(data=dfhenry, x=xs, y=ys, legend=True, sizes=(20, 2000))\nsns.scatterplot(data=dfsarah[:300], x=xs, y=ys, legend=True, sizes=(20, 2000))\nsns.scatterplot(data=dfjoe, x=xs, y=ys, legend=True, sizes=(20, 2000))\n#sns.scatterplot(data=dfgeorge, x=xs, y=ys, legend=True, sizes=(20, 2000))\n\n# show the graph\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:16.792733Z","iopub.execute_input":"2022-09-16T21:40:16.793104Z","iopub.status.idle":"2022-09-16T21:40:17.062077Z","shell.execute_reply.started":"2022-09-16T21:40:16.793066Z","shell.execute_reply":"2022-09-16T21:40:17.060680Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dflarge=dfhenry.nlargest(100,'conf')\ndfsmall=dfhenry.nsmallest(100,'conf')\n\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\n# Control figure size for this notebook:\nplt.rcParams['figure.figsize'] = [16, 16]\n\n# data \nxs='avgmidDistXCenter'\nys='avgmidDistYCenter'\n \n# use the scatterplot function to build the bubble map\nsns.scatterplot(data=dflarge, x=xs, y=ys, legend=True, sizes=(20, 1000))\nsns.scatterplot(data=dfsmall, x=xs, y=ys, legend=True, sizes=(20, 1000))\n\n# show the graph\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:17.063898Z","iopub.execute_input":"2022-09-16T21:40:17.065202Z","iopub.status.idle":"2022-09-16T21:40:17.348160Z","shell.execute_reply.started":"2022-09-16T21:40:17.065151Z","shell.execute_reply":"2022-09-16T21:40:17.347244Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# This is our best shot\ntry1=dfhenry.nlargest(1,'conf')\ntry1['conf']","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:17.349825Z","iopub.execute_input":"2022-09-16T21:40:17.350169Z","iopub.status.idle":"2022-09-16T21:40:17.360859Z","shell.execute_reply.started":"2022-09-16T21:40:17.350136Z","shell.execute_reply":"2022-09-16T21:40:17.359696Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"listx=[\"x1\", \"x2\",\"x3\", \"x4\",\"x5\", \"x6\",\"x7\", \"x8\",\"x9\", \"x10\",\"x11\", \"x12\",\"x13\", \"x14\",\"x15\"]\n## Would run through test to see if it can be closer to observaitions... makes things worse\nflag = False\nwhile (flag):\n flag = False\n for x in listx:\n tester=int(try1[x])\n testeravgmid=int(try1['midx'])\n diff=tester - testeravgmid\n if (abs(diff)>7.5):\n if (diff>0):\n try1[x]=round(tester-1,0)\n else:\n try1[x]=round(tester+1,0)\n print (\"Fixing X Over \"+str(x)+\" now \"+str(try1[x]))\n flag = True\n break\n elif (abs(diff)<3):\n if (diff>0):\n try1[x]=round((tester+1),0)\n else:\n try1[x]=round((tester-1),0)\n print (\"Fixing X Under \"+str(x)+\" now \"+str(try1[x]))\n flag = True\n break\n try1['midx']=(try1['x1']+try1['x2']+try1['x3']+try1['x4']+try1['x5']+try1['x6']+try1['x7']+try1['x8']+try1['x9']+try1['x10']+try1['x11']+try1['x12']+try1['x13']+try1['x14']+try1['x15'])/15\nflag = False\nlisty=[\"y1\", \"y2\",\"y3\", \"y4\",\"y5\", \"y6\",\"y7\", \"y8\",\"y9\", \"y10\",\"y11\", \"y12\",\"y13\", \"y14\",\"y15\"]\nwhile (flag):\n flag = False\n for x in listy:\n tester=int(try1[x])\n testeravgmid=int(try1['midy'])\n diff=tester - testeravgmid\n if (abs(diff)>7.5):\n if (diff>0):\n try1[x]=round((tester)+1,0)\n else:\n if (round((tester)-1,0)<0):\n try1[x]=11\n else:\n try1[x]=round((tester)-1,0)\n elif (abs(diff)<3):\n if (diff>0):\n try1[x]=round((tester)+1,0)\n else:\n if (round((tester)-1,0)<0):\n try1[x]=10\n else:\n try1[x]=round((tester)-1,0)\n print (\"Fixing Y Under \"+str(x)+\" now \"+str(try1[x]))\n flag = True\n try1['midy']=(try1['y1']+try1['y2']+try1['y3']+try1['y4']+try1['y5']+try1['y6']+try1['y7']+try1['y8']+try1['y9']+try1['y10']+try1['y11']+try1['y12']+try1['y13']+try1['y14']+try1['y15'])/15\n\n \n\n#check if x is greater than 4 less than 7 from center X\n#check if y is greater than 4 less than 7 from center y\n\n\n#check if x is greater than 4 less than 7 from midpoint X\n#check if y is greater than 4 less than 7 from midpoint y\n\n#check no duplicates","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:17.362842Z","iopub.execute_input":"2022-09-16T21:40:17.363672Z","iopub.status.idle":"2022-09-16T21:40:17.382822Z","shell.execute_reply.started":"2022-09-16T21:40:17.363639Z","shell.execute_reply":"2022-09-16T21:40:17.381739Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def checkduplicates(test):\n listcheck=[]\n for i in range (0,15):\n added=str(test[\"pitches\"][i][\"x\"])+\" \"+str(test[\"pitches\"][i][\"y\"])\n if added in listcheck:\n print (added +\" at \"+str(i))\n listcheck.append(added)\n\n if len(listcheck) == len(set(listcheck)):\n return False\n else:\n return True\n\ndef pandatosubmit(try1,test):\n for t in range(0, 15): #14 or 15\n test[\"pitches\"][t][\"x\"]=int(try1.iloc[0]['x'+str(t+1)])\n test[\"pitches\"][t][\"y\"]=int(try1.iloc[0]['y'+str(t+1)])\n return baseball2(test)\n\ndef addsubmit(test,dfa,check):\n if \"henry\" in check:\n check1=check.split(\"Confidence: \")\n check2=check1[1].split('.\", 200]}')\n score=float(check2[0])\n name=\"henry\"\n dfa.loc[len(dfa.index)] = [test['pitches'][0]['x'], test['pitches'][0]['y'],\n test['pitches'][1]['x'], test['pitches'][1]['y'],\n test['pitches'][2]['x'], test['pitches'][2]['y'],\n test['pitches'][3]['x'], test['pitches'][3]['y'],\n test['pitches'][4]['x'], test['pitches'][4]['y'],\n test['pitches'][5]['x'], test['pitches'][5]['y'],\n test['pitches'][6]['x'], test['pitches'][6]['y'],\n test['pitches'][7]['x'], test['pitches'][7]['y'],\n test['pitches'][8]['x'], test['pitches'][8]['y'],\n test['pitches'][9]['x'], test['pitches'][9]['y'],\n test['pitches'][10]['x'], test['pitches'][10]['y'],\n test['pitches'][11]['x'], test['pitches'][11]['y'],\n test['pitches'][12]['x'], test['pitches'][12]['y'],\n test['pitches'][13]['x'], test['pitches'][13]['y'],\n test['pitches'][14]['x'], test['pitches'][14]['y'],\n '0','0',float(check2[0]),name]\n return dfa\n else:\n return False","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:17.384469Z","iopub.execute_input":"2022-09-16T21:40:17.384829Z","iopub.status.idle":"2022-09-16T21:40:17.402719Z","shell.execute_reply.started":"2022-09-16T21:40:17.384796Z","shell.execute_reply":"2022-09-16T21:40:17.401718Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pandatosubmit(try1,test)","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:17.404510Z","iopub.execute_input":"2022-09-16T21:40:17.405214Z","iopub.status.idle":"2022-09-16T21:40:17.722638Z","shell.execute_reply.started":"2022-09-16T21:40:17.405180Z","shell.execute_reply":"2022-09-16T21:40:17.721353Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Some helper Functions\n\npandatosubmit(try1,test)\nprint (baseball2(test))\ncheckduplicates(test)","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:17.724916Z","iopub.execute_input":"2022-09-16T21:40:17.725824Z","iopub.status.idle":"2022-09-16T21:40:18.369880Z","shell.execute_reply.started":"2022-09-16T21:40:17.725771Z","shell.execute_reply":"2022-09-16T21:40:18.368499Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"t=9\ntest[\"pitches\"][t][\"x\"]=9\ntest[\"pitches\"][t][\"y\"]=21\nprint (baseball2(test))\ncheckduplicates(test)","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:18.371559Z","iopub.execute_input":"2022-09-16T21:40:18.372713Z","iopub.status.idle":"2022-09-16T21:40:18.853977Z","shell.execute_reply.started":"2022-09-16T21:40:18.372666Z","shell.execute_reply":"2022-09-16T21:40:18.852519Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"listx=[\"x1\", \"x2\",\"x3\", \"x4\",\"x5\", \"x6\",\"x7\", \"x8\",\"x9\", \"x10\",\"x11\", \"x12\",\"x13\", \"x14\",\"x15\"]\nfor x in listx:\n dfa[x+\"midDist\"]=abs(dfa[x]-dfa['midx'])\n \nlisty=[\"y1\", \"y2\",\"y3\", \"y4\",\"y5\", \"y6\",\"y7\", \"y8\",\"y9\", \"y10\",\"y11\", \"y12\",\"y13\", \"y14\",\"y15\"]\nfor x in listy:\n dfa[x+\"midDist\"]=abs(dfa[x]-dfa['midy'])\n \ndfa['avgmidDistX']=(dfa['x1midDist']+dfa['x2midDist']+dfa['x3midDist']+dfa['x4midDist']+dfa['x5midDist']+dfa['x6midDist']+dfa['x7midDist']+dfa['x8midDist']+dfa['x9midDist']+dfa['x10midDist']+dfa['x11midDist']+dfa['x12midDist']+dfa['x13midDist']+dfa['x14midDist']+dfa['x15midDist'])/15\ndfa['avgmidDistY']=(dfa['y1midDist']+dfa['y2midDist']+dfa['y3midDist']+dfa['y4midDist']+dfa['y5midDist']+dfa['y6midDist']+dfa['y7midDist']+dfa['y8midDist']+dfa['y9midDist']+dfa['y10midDist']+dfa['y11midDist']+dfa['y12midDist']+dfa['y13midDist']+dfa['y14midDist']+dfa['y15midDist'])/15","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:18.856225Z","iopub.execute_input":"2022-09-16T21:40:18.856988Z","iopub.status.idle":"2022-09-16T21:40:18.895143Z","shell.execute_reply.started":"2022-09-16T21:40:18.856940Z","shell.execute_reply":"2022-09-16T21:40:18.893911Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#dfb=addsubmit(test,dfa,pandatosubmit(try1,test))","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:18.896421Z","iopub.execute_input":"2022-09-16T21:40:18.896979Z","iopub.status.idle":"2022-09-16T21:40:18.901012Z","shell.execute_reply.started":"2022-09-16T21:40:18.896946Z","shell.execute_reply":"2022-09-16T21:40:18.900089Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"if \"henry\" in check:\n check1=check.split(\"Confidence: \")\n check2=check1[1].split('.\", 200]}')\n score=float(check2[0])\n #dfplots=scoring(dfplots, test, score)\n if (score>.4):\n print (\"Hi Henry\")\n henrylist.append(test.copy())\n name=\"henry\"\n\n dfa.loc[len(dfa.index)] = [test['pitches'][0]['x'], test['pitches'][0]['y'],\n test['pitches'][1]['x'], test['pitches'][1]['y'],\n test['pitches'][2]['x'], test['pitches'][2]['y'],\n test['pitches'][3]['x'], test['pitches'][3]['y'],\n test['pitches'][4]['x'], test['pitches'][4]['y'],\n test['pitches'][5]['x'], test['pitches'][5]['y'],\n test['pitches'][6]['x'], test['pitches'][6]['y'],\n test['pitches'][7]['x'], test['pitches'][7]['y'],\n test['pitches'][8]['x'], test['pitches'][8]['y'],\n test['pitches'][9]['x'], test['pitches'][9]['y'],\n test['pitches'][10]['x'], test['pitches'][10]['y'],\n test['pitches'][11]['x'], test['pitches'][11]['y'],\n test['pitches'][12]['x'], test['pitches'][12]['y'],\n test['pitches'][13]['x'], test['pitches'][13]['y'],\n test['pitches'][14]['x'], test['pitches'][14]['y'],\n '0','0',float(check2[0]),name]","metadata":{"execution":{"iopub.status.busy":"2022-09-11T23:22:25.859038Z","iopub.execute_input":"2022-09-11T23:22:25.860024Z","iopub.status.idle":"2022-09-11T23:22:25.867166Z","shell.execute_reply.started":"2022-09-11T23:22:25.859978Z","shell.execute_reply":"2022-09-11T23:22:25.865549Z"}}},{"cell_type":"markdown","source":"# joe, george, sarah, henry\nfor i in range(1,10000):\n #Make a random sample\n for t in range(0, 15):\n test[\"pitches\"][t][\"x\"]=randrange(29)\n test[\"pitches\"][t][\"y\"]=randrange(29)\n \n #Split the result\n check=baseball2(test)\n\n if \"henry\" in check:\n check1=check.split(\"Confidence: \")\n check2=check1[1].split('.\", 200]}')\n score=float(check2[0])\n dfplots=scoring(dfplots, test, score)\n elif \"sarah\" in check:\n check1=check.split(\"liklihood of \")\n check2=check1[1].split('\", 200]}')\n score=float(check2[0])*-1\n dfplots=scoring(dfplots, test, score)\n elif \"Not 15 unique pitches.\" in check:\n bob=1\n else:\n print (\"Something went wrong \"+str(check))\n ","metadata":{"execution":{"iopub.status.busy":"2022-09-11T23:22:35.178173Z","iopub.execute_input":"2022-09-11T23:22:35.178987Z","iopub.status.idle":"2022-09-11T23:22:39.986529Z","shell.execute_reply.started":"2022-09-11T23:22:35.178936Z","shell.execute_reply":"2022-09-11T23:22:39.984363Z"}}},{"cell_type":"markdown","source":"# joe, george, sarah, henry\nfor i in range(1,10000):\n #Make a random sample\n for t in range(0, 15):\n test[\"pitches\"][t][\"x\"]=randrange(29)\n test[\"pitches\"][t][\"y\"]=randrange(29)\n \n #Split the result\n check=baseball2(test)\n\n if \"henry\" in check:\n check1=check.split(\"Confidence: \")\n check2=check1[1].split('.\", 200]}')\n score=float(check2[0])\n dfplots=scoring(dfplots, test, score)\n elif \"sarah\" in check:\n check1=check.split(\"liklihood of \")\n check2=check1[1].split('\", 200]}')\n score=float(check2[0])*-1\n dfplots=scoring(dfplots, test, score)\n elif \"Not 15 unique pitches.\" in check:\n bob=1\n else:\n print (\"Something went wrong \"+str(check))\n ","metadata":{"execution":{"iopub.status.busy":"2022-09-11T23:22:39.988245Z","iopub.status.idle":"2022-09-11T23:22:39.989224Z","shell.execute_reply.started":"2022-09-11T23:22:39.988914Z","shell.execute_reply":"2022-09-11T23:22:39.988945Z"}}},{"cell_type":"code","source":"dfplots['avg'] = dfplots['conf']/dfplots['count']","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:18.902350Z","iopub.execute_input":"2022-09-16T21:40:18.903549Z","iopub.status.idle":"2022-09-16T21:40:18.914303Z","shell.execute_reply.started":"2022-09-16T21:40:18.903514Z","shell.execute_reply":"2022-09-16T21:40:18.913268Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dflarge=dfplots.nlargest(15,'avg')","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:18.915673Z","iopub.execute_input":"2022-09-16T21:40:18.916914Z","iopub.status.idle":"2022-09-16T21:40:18.928071Z","shell.execute_reply.started":"2022-09-16T21:40:18.916874Z","shell.execute_reply":"2022-09-16T21:40:18.926831Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import scipy.stats as stats\n\n# If you need the model coefficients stored in variables for later use, do:\nxs= dfplots['x']\nys= dfplots['y']\nmodel = stats.linregress( xs, ys )\nbeta0 = model.intercept\nbeta1 = model.slope\n\n# If you just need to see the coefficients (and some other related data),\n# do this alone:\nstats.linregress( xs, ys )","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:18.929469Z","iopub.execute_input":"2022-09-16T21:40:18.929844Z","iopub.status.idle":"2022-09-16T21:40:18.943752Z","shell.execute_reply.started":"2022-09-16T21:40:18.929812Z","shell.execute_reply":"2022-09-16T21:40:18.942410Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Massive Function to try and figure out \n\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\n# Control figure size for this notebook:\nplt.rcParams['figure.figsize'] = [8, 8]\n\n\ncheck=\"\"\nblink=0\nbestone=.3\n\nwhile (1 ==1):\n for t in range(0, 15):\n test[\"pitches\"][t][\"x\"]=randrange(29)\n test[\"pitches\"][t][\"y\"]=randrange(29)\n check=baseball(0,1,test)\n #print (check)\n \n \n if \"henry\" in check:\n #sns.scatterplot(data=plots, x=\"x\", y=\"y\", legend=False, sizes=(20, 2000))\n check1=check.split(\"Confidence: \")\n check2=check1[1].split('.\", 200]}')\n if (float(check2[0])>.46):\n blink=blink+1\n print (check)\n plots=pd.DataFrame.from_dict(test['pitches'])\n sns.scatterplot(data=plots, x=\"x\", y=\"y\", legend=False, sizes=(20, 2000),alpha=0.3)\n if (blink%10==0):\n plt.show()\n if (float(check2[0])>bestone):\n extra = copy.deepcopy(test)\n bestone = float(check2[0])\n print (\"NewBest \"+str(bestone))\n if (float(check2[0])>.6):\n print (test)\n break","metadata":{}},{"cell_type":"code","source":"plt.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:18.945251Z","iopub.execute_input":"2022-09-16T21:40:18.946131Z","iopub.status.idle":"2022-09-16T21:40:18.951091Z","shell.execute_reply.started":"2022-09-16T21:40:18.946097Z","shell.execute_reply":"2022-09-16T21:40:18.950057Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Different Equations\ndef equationAdd(x,y,test,coordinateList):\n for t in range(0, 15): #14 or 15\n test[\"pitches\"][t][\"x\"]=(x+t)%30\n test[\"pitches\"][t][\"y\"]=(y+t)%30\n check= (baseball(x,y,test))\n if \"henry\" in check:\n #print (check)\n check1=check.split(\"Confidence: \")\n check2=check1[1].split('.\", 200]}')\n coordinateList.loc[len(coordinateList.index)] = [x, y,float(check2[0])]\n\ndef equationMixA(x,y,test,coordinateList):\n for t in range(0, 15): #14 or 15\n test[\"pitches\"][t][\"x\"]=(x-t)%30\n test[\"pitches\"][t][\"y\"]=(y+t)%30\n check= (baseball(x,y,test))\n if \"henry\" in check:\n print (check)\n check1=check.split(\"Confidence: \")\n check2=check1[1].split('.\", 200]}')\n coordinateList.loc[len(coordinateList.index)] = [x, y,float(check2[0])]\n \ndef equationMixB(x,y,test,coordinateList):\n for t in range(0, 15): #14 or 15\n test[\"pitches\"][t][\"x\"]=(x+t)%30\n test[\"pitches\"][t][\"y\"]=(y-t)%30\n check= (baseball(x,y,test))\n if \"henry\" in check:\n print (check)\n check1=check.split(\"Confidence: \")\n check2=check1[1].split('.\", 200]}')\n coordinateList.loc[len(coordinateList.index)] = [x, y,float(check2[0])]\n\ndef equationMixC(x,y,test,coordinateList):\n for t in range(0, 15): #14 or 15\n test[\"pitches\"][t][\"x\"]=(3*x+t)%30\n test[\"pitches\"][t][\"y\"]=(y-t)%30\n check= (baseball(x,y,test))\n if \"henry\" in check:\n print (check)\n check1=check.split(\"Confidence: \")\n check2=check1[1].split('.\", 200]}')\n coordinateList.loc[len(coordinateList.index)] = [x, y,float(check2[0])]\n \ndef equationMixD(x,y,test,coordinateList):\n for t in range(0, 15): #14 or 15\n test[\"pitches\"][t][\"x\"]=(x+t)%30\n test[\"pitches\"][t][\"y\"]=(3*y-t)%30\n check= (baseball(x,y,test))\n if \"henry\" in check:\n print (check)\n check1=check.split(\"Confidence: \")\n check2=check1[1].split('.\", 200]}')\n coordinateList.loc[len(coordinateList.index)] = [x, y,float(check2[0])]\n \ndef equationMixLin(x,y,test,coordinateList):\n for t in range(0, 15): #14 or 15\n xs=t+x\n test[\"pitches\"][t][\"x\"]=(xs)%30\n test[\"pitches\"][t][\"y\"]=(-xs+29)%30\n check= (baseball(x,y,test))\n if \"henry\" in check:\n #print (check)\n check1=check.split(\"Confidence: \")\n check2=check1[1].split('.\", 200]}')\n coordinateList.loc[len(coordinateList.index)] = [x, y,float(check2[0])] ","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:18.952702Z","iopub.execute_input":"2022-09-16T21:40:18.953042Z","iopub.status.idle":"2022-09-16T21:40:18.975972Z","shell.execute_reply.started":"2022-09-16T21:40:18.953011Z","shell.execute_reply":"2022-09-16T21:40:18.974701Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Sets up just for \ndef foursquare(t,test,bestconf,step):\n \n improvement = False\n baseline = copy.deepcopy(test)\n best = copy.deepcopy(test)\n \n #top Right\n answer=cornershuffle(t,test,bestconf,step,1,1)\n if(answer[0]):\n bestconf = answer[1]\n improvement = True\n best = copy.deepcopy(test)\n \n #top \n answer=cornershuffle(t,test,bestconf,step,1,0)\n if(answer[0]):\n bestconf = answer[1]\n improvement = True\n best = copy.deepcopy(test)\n \n #top Left\n answer=cornershuffle(t,test,bestconf,step,1,-1)\n if(answer[0]):\n bestconf = answer[1]\n improvement = True\n best = copy.deepcopy(test)\n \n #right \n answer=cornershuffle(t,test,bestconf,step,0,1)\n if(answer[0]):\n bestconf = answer[1]\n improvement = True\n best = copy.deepcopy(test)\n \n #bottom Right\n answer=cornershuffle(t,test,bestconf,step,-1,1)\n if(answer[0]):\n bestconf = answer[1]\n improvement = True\n best = copy.deepcopy(test)\n \n #Bottom \n answer=cornershuffle(t,test,bestconf,step,0,-1)\n if(answer[0]):\n bestconf = answer[1]\n improvement = True\n best = copy.deepcopy(test)\n \n #bottom Left\n answer=cornershuffle(t,test,bestconf,step,-1,-1)\n if(answer[0]):\n bestconf = answer[1]\n improvement = True\n best = copy.deepcopy(test)\n \n #left \n answer=cornershuffle(t,test,bestconf,step,-1,0)\n if(answer[0]):\n bestconf = answer[1]\n improvement = True\n best = copy.deepcopy(test)\n \n if(improvement):\n test = copy.deepcopy(best)\n #print (\"improvement\")\n else:\n test = copy.deepcopy(baseline)\n #print (\"no improvement\")\n \n return improvement, bestconf","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:18.977604Z","iopub.execute_input":"2022-09-16T21:40:18.977958Z","iopub.status.idle":"2022-09-16T21:40:18.992984Z","shell.execute_reply.started":"2022-09-16T21:40:18.977920Z","shell.execute_reply":"2022-09-16T21:40:18.992017Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def betterfit(check, bestconf):\n if \"henry\" in check:\n #print (check)\n check1=check.split(\"Confidence: \")\n check2=check1[1].split('.\", 200]}')\n if (float(check2[0])>bestconf):\n return float(check2[0])\n else:\n return False\n else:\n return False\n\ndef xshuffle(t,test,bestconf,step):\n x=test[\"pitches\"][t][\"x\"]\n test[\"pitches\"][t][\"x\"] = (x+step)%30\n score=betterfit(baseball(2,3,test),bestconf)\n if (score > bestconf):\n bestconf=score\n return True, score\n test[\"pitches\"][t][\"x\"] = (x-step)%30\n score=betterfit(baseball(2,3,test),bestconf)\n if (score > bestconf):\n bestconf=score\n return True,score\n test[\"pitches\"][t][\"x\"] = (x)%30\n return False, bestconf\n\ndef yshuffle(t,test,bestconf,step):\n y=test[\"pitches\"][t][\"y\"]\n test[\"pitches\"][t][\"y\"] = (y+step)%30\n score=betterfit(baseball(2,3,test),bestconf)\n if (score > bestconf):\n bestconf=score\n return True, score\n test[\"pitches\"][t][\"y\"] = (y-step)%30\n score=betterfit(baseball(2,3,test),bestconf)\n if (score > bestconf):\n bestconf=score\n return True,score\n test[\"pitches\"][t][\"y\"] = (y)%30\n return False, bestconf\n\ndef cornershuffle(t,test,bestconf,step,x1,y1):\n x=test[\"pitches\"][t][\"x\"]\n y=test[\"pitches\"][t][\"y\"]\n xstep=step*x1\n ystep=step*y1\n test[\"pitches\"][t][\"y\"] = (y+ystep)%30\n test[\"pitches\"][t][\"x\"] = (x+xstep)%30\n score=betterfit(baseball(2,3,test),bestconf)\n if (score > bestconf):\n bestconf=score\n return True, score\n test[\"pitches\"][t][\"y\"] = (y-ystep)%30\n test[\"pitches\"][t][\"x\"] = (x-xstep)%30\n score=betterfit(baseball(2,3,test),bestconf)\n if (score > bestconf):\n bestconf=score\n return True,score\n test[\"pitches\"][t][\"y\"] = (y)%30\n test[\"pitches\"][t][\"x\"] = (x)%30\n return False, bestconf\n\ndef fourcorners(t,test,bestconf,step):\n \n improvement = False\n baseline = copy.deepcopy(test)\n best = copy.deepcopy(test)\n \n #top Right\n answer=cornershuffle(t,test,bestconf,step,1,1)\n if(answer[0]):\n bestconf = answer[1]\n improvement = True\n best = copy.deepcopy(test)\n \n #top Left\n answer=cornershuffle(t,test,bestconf,step,1,-1)\n if(answer[0]):\n bestconf = answer[1]\n improvement = True\n best = copy.deepcopy(test)\n \n #bottom Right\n answer=cornershuffle(t,test,bestconf,step,-1,1)\n if(answer[0]):\n bestconf = answer[1]\n improvement = True\n best = copy.deepcopy(test)\n \n #bottom Left\n answer=cornershuffle(t,test,bestconf,step,-1,-1)\n if(answer[0]):\n bestconf = answer[1]\n improvement = True\n best = copy.deepcopy(test)\n \n if(improvement):\n test = copy.deepcopy(best)\n else:\n test = copy.deepcopy(baseline)\n \n return improvement, bestconf","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:18.994306Z","iopub.execute_input":"2022-09-16T21:40:18.994700Z","iopub.status.idle":"2022-09-16T21:40:19.019925Z","shell.execute_reply.started":"2022-09-16T21:40:18.994668Z","shell.execute_reply":"2022-09-16T21:40:19.018612Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:19.021895Z","iopub.execute_input":"2022-09-16T21:40:19.022661Z","iopub.status.idle":"2022-09-16T21:40:19.035239Z","shell.execute_reply.started":"2022-09-16T21:40:19.022619Z","shell.execute_reply":"2022-09-16T21:40:19.034378Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Function to check for things close by; Explore this more\n\nbestconf= .3\nstep = 3\n\n#addsubmit(test,dfa,check)\n\nfor run in range (0,50):\n if (step<1):\n print (\"At max\")\n break\n \n baseline=bestconf\n \n for i in range(0,15):\n answer=foursquare(i,test,bestconf,step)\n if(answer[0]):\n check=baseball2(test)\n dfa=addsubmit(test,dfa,check)\n bestconf=answer[1]\n print(\"New Best \"+str(bestconf))\n besttest = copy.deepcopy(test)\n print (baseball(2,3,test))\n if (baseline==bestconf):\n step=step-1\n print(\"down to step \"+str(step))\n else:\n step=5\n \n\nbaseball2(test)\n","metadata":{"execution":{"iopub.status.busy":"2022-09-13T12:44:52.881773Z","iopub.execute_input":"2022-09-13T12:44:52.882124Z","iopub.status.idle":"2022-09-13T12:45:06.295427Z","shell.execute_reply.started":"2022-09-13T12:44:52.882092Z","shell.execute_reply":"2022-09-13T12:45:06.281903Z"}}},{"cell_type":"markdown","source":"import matplotlib.pyplot as plt\nimport seaborn as sns\n\n# Control figure size for this notebook:\nplt.rcParams['figure.figsize'] = [8, 8]\n\n# data \ndata = trya\ndatab = tryb\ndatac = tryc\ndatad = tryd\n \n# use the scatterplot function to build the bubble map\nsns.scatterplot(data=data, x=\"x\", y=\"y\", legend=False, sizes=(20, 2000))\nsns.scatterplot(data=datab, x=\"x\", y=\"y\", legend=False, sizes=(20, 2000))\nsns.scatterplot(data=datac, x=\"x\", y=\"y\", legend=False, sizes=(20, 2000))\nsns.scatterplot(data=datad, x=\"x\", y=\"y\", legend=False, sizes=(20, 2000))\n\n# show the graph\nplt.show()","metadata":{}},{"cell_type":"code","source":"def generatelist(x,y,newlist):\n\n for t in range(0, 15):\n x=(x+t)%30\n y=(y+t)%30\n if (1==3):\n print (\"error\")#x and y are in newlist +1\n else:\n newlist.loc[len(newlist.index)] = [x, y, 0] \n return newlist\n\ndef generatelistA(x,y,newlist):\n\n for t in range(0, 15):\n x=(x-t)%30\n y=(y+t)%30\n if (1==3):\n print (\"error\")#x and y are in newlist +1\n else:\n newlist.loc[len(newlist.index)] = [x, y, 0] \n return newlist\n\ndef generatelistB(x,y,newlist):\n\n for t in range(0, 15):\n x=(x+t)%30\n y=(y-t)%30\n if (1==3):\n print (\"error\")#x and y are in newlist +1\n else:\n newlist.loc[len(newlist.index)] = [x, y, 0] \n return newlist\n\ndef generatelistC(x,y,newlist):\n\n for t in range(0, 15):\n x=(3*x+t)%30\n y=(y-t)%30\n if (1==3):\n print (\"error\")#x and y are in newlist +1\n else:\n newlist.loc[len(newlist.index)] = [x, y, 0] \n return newlist\n\ndef generatelistD(x,y,newlist):\n\n for t in range(0, 15):\n x=(x+t)%30\n y=(3*y-t)%30\n if (1==3):\n print (\"error\")#x and y are in newlist +1\n else:\n newlist.loc[len(newlist.index)] = [x, y, 0] \n return newlist\n\ndef generateMixLin(x,y,newlist):\n\n for t in range(0, 15):\n xs=t+x\n x=(xs)%30\n y=(-xs+29)%30\n if (1==3):\n print (\"error\")#x and y are in newlist +1\n else:\n newlist.loc[len(newlist.index)] = [x, y, 0] \n return newlist","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:19.036834Z","iopub.execute_input":"2022-09-16T21:40:19.037456Z","iopub.status.idle":"2022-09-16T21:40:19.052921Z","shell.execute_reply.started":"2022-09-16T21:40:19.037417Z","shell.execute_reply":"2022-09-16T21:40:19.051836Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# makes some old stuff\ncoordinatelist = {'x':[],\n 'y':[],\n 'count':[]\n }\n\ncoordinatelist = pd.DataFrame(coordinatelist)\ncoordinatelist=generatelistA(22,7,coordinatelist)\ncoordinatelist","metadata":{"execution":{"iopub.status.busy":"2022-09-12T04:02:04.025780Z","iopub.execute_input":"2022-09-12T04:02:04.026752Z","iopub.status.idle":"2022-09-12T04:02:04.067769Z","shell.execute_reply.started":"2022-09-12T04:02:04.026708Z","shell.execute_reply":"2022-09-12T04:02:04.066693Z"}}},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"import pickle\nmodel_file = \"top.pickle\"\nwith open(model_file,'wb') as f:\n pickle.dump(coordinatelist, f)","metadata":{"execution":{"iopub.status.busy":"2022-09-13T18:58:02.802816Z","iopub.execute_input":"2022-09-13T18:58:02.803251Z","iopub.status.idle":"2022-09-13T18:58:02.957472Z","shell.execute_reply.started":"2022-09-13T18:58:02.803216Z","shell.execute_reply":"2022-09-13T18:58:02.955873Z"}}},{"cell_type":"markdown","source":"import scipy.stats as stats\n\n# If you need the model coefficients stored in variables for later use, do:\nxs= top['x']\nys= top['y']\nmodel = stats.linregress( xs, ys )\nbeta0 = model.intercept\nbeta1 = model.slope\n\n# If you just need to see the coefficients (and some other related data),\n# do this alone:\nstats.linregress( xs, ys )","metadata":{"execution":{"iopub.status.busy":"2022-09-11T23:24:04.696462Z","iopub.execute_input":"2022-09-11T23:24:04.697289Z","iopub.status.idle":"2022-09-11T23:24:04.720496Z","shell.execute_reply.started":"2022-09-11T23:24:04.697238Z","shell.execute_reply":"2022-09-11T23:24:04.718649Z"}}},{"cell_type":"markdown","source":"import matplotlib.pyplot as plt\nimport seaborn as sns\n\n# Control figure size for this notebook:\nplt.rcParams['figure.figsize'] = [8, 8]\n\n# data \ndata = top\n \n# use the scatterplot function to build the bubble map\nsns.scatterplot(data=data, x=\"x\", y=\"y\", size=\"count\", legend=False, sizes=(20, 2000))\n\n# show the graph\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-11T23:24:24.000059Z","iopub.execute_input":"2022-09-11T23:24:24.000478Z","iopub.status.idle":"2022-09-11T23:24:24.040918Z","shell.execute_reply.started":"2022-09-11T23:24:24.000442Z","shell.execute_reply":"2022-09-11T23:24:24.038159Z"}}},{"cell_type":"code","source":"# Smaller Group\ntop = dflarge\n\nt=0\nfor row in top.itertuples():\n x=row.x\n y=row.y\n test[\"pitches\"][t][\"x\"]=row.x\n test[\"pitches\"][t][\"y\"]=row.y\n t=t+1\n \ntest","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:19.054465Z","iopub.execute_input":"2022-09-16T21:40:19.055062Z","iopub.status.idle":"2022-09-16T21:40:19.073502Z","shell.execute_reply.started":"2022-09-16T21:40:19.055025Z","shell.execute_reply":"2022-09-16T21:40:19.072498Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\nwith open(f\"{dir_name}/baseball/test_pitch.json\", \"r\") as f:\n test = json.load(f)\n\ntry:\n r = requests.post(\"https://bball.fly.dev/score\", json=test)\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:19.074991Z","iopub.execute_input":"2022-09-16T21:40:19.075436Z","iopub.status.idle":"2022-09-16T21:40:19.407204Z","shell.execute_reply.started":"2022-09-16T21:40:19.075387Z","shell.execute_reply":"2022-09-16T21:40:19.406392Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"baseball\", \"ICEXDEILLP5VXE2IQXXRDULBDGJ9PXQIATAHNTCFQDJ{AOE8YZA3WCXMYLH7GD9A0768MRT39BFG}2ID267HHO4U6TPI28LGLKIDZUXP36JCMK90EXFX1VPKF4EUPHL6\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:19.427778Z","iopub.execute_input":"2022-09-16T21:40:19.428621Z","iopub.status.idle":"2022-09-16T21:40:19.438949Z","shell.execute_reply.started":"2022-09-16T21:40:19.428575Z","shell.execute_reply":"2022-09-16T21:40:19.437825Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n\n# FORENSICS (id: forensics) -- 300 Points\n\nPretty straight forward\n\n\n\n\n\n","metadata":{}},{"cell_type":"code","source":"!ls /kaggle/input/ai-village-ctf/forensics","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:19.440612Z","iopub.execute_input":"2022-09-16T21:40:19.440981Z","iopub.status.idle":"2022-09-16T21:40:20.551118Z","shell.execute_reply.started":"2022-09-16T21:40:19.440945Z","shell.execute_reply":"2022-09-16T21:40:20.549570Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from tensorflow import keras\nmodel = keras.models.load_model('/kaggle/input/ai-village-ctf/forensics/forensics.h5')","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:20.552914Z","iopub.execute_input":"2022-09-16T21:40:20.553387Z","iopub.status.idle":"2022-09-16T21:40:28.517570Z","shell.execute_reply.started":"2022-09-16T21:40:20.553341Z","shell.execute_reply":"2022-09-16T21:40:28.516367Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.summary()","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:28.518916Z","iopub.execute_input":"2022-09-16T21:40:28.519227Z","iopub.status.idle":"2022-09-16T21:40:28.547232Z","shell.execute_reply.started":"2022-09-16T21:40:28.519198Z","shell.execute_reply":"2022-09-16T21:40:28.545559Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\ntry:\n r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"forensics\", \"submission\": \"FLAG{F0R3NS1C5_XD}\"}))\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:28.549075Z","iopub.execute_input":"2022-09-16T21:40:28.550773Z","iopub.status.idle":"2022-09-16T21:40:28.696179Z","shell.execute_reply.started":"2022-09-16T21:40:28.550723Z","shell.execute_reply":"2022-09-16T21:40:28.694922Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"forensics\", \"3GN1BT34W0YX8YUN4LG5Z6H56GM73MAIL}P2W3HQZF8XVNFMNHS6JQR5AZ4N1VBDFCFQMF21HUJ8R4O1IL6NYFMO7S1VF1FM0AB78HGBJMVYA{BC2_UPFWE9RSSWEZKH\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:28.697799Z","iopub.execute_input":"2022-09-16T21:40:28.698997Z","iopub.status.idle":"2022-09-16T21:40:28.710930Z","shell.execute_reply.started":"2022-09-16T21:40:28.698945Z","shell.execute_reply":"2022-09-16T21:40:28.709671Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n\n# Token (id: token) -- 500 Points\n\nI tried to solve this a number of ways. But it turned out that through Bruteforce I was looking in the wrong areas.","metadata":{}},{"cell_type":"code","source":"!ls /kaggle/input/ai-village-ctf/token","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:28.712621Z","iopub.execute_input":"2022-09-16T21:40:28.712951Z","iopub.status.idle":"2022-09-16T21:40:29.782585Z","shell.execute_reply.started":"2022-09-16T21:40:28.712921Z","shell.execute_reply":"2022-09-16T21:40:29.780925Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df = pd.read_csv ('/kaggle/input/ai-village-ctf/token/test.csv')\ndf[760:]","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:29.784896Z","iopub.execute_input":"2022-09-16T21:40:29.785880Z","iopub.status.idle":"2022-09-16T21:40:29.817405Z","shell.execute_reply.started":"2022-09-16T21:40:29.785815Z","shell.execute_reply":"2022-09-16T21:40:29.815887Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Briteforced- Like bruteforce but brighter\noldhigh = 0.0075\n#nothing up to 180; 762-795\n#Try starting at 762 before the weirdness\n#Trying starting at 1130\nfor i1 in range(1130,1752):\n for i2 in range(0,1752):\n data = json.dumps(\n {\n \"line1\": i1,\n \"line2\": i2\n }\n )\n try:\n r = requests.post(\"https://token.fly.dev/score\", data=data)\n check=r.text\n check1=check.split(\": \")\n check2=check1[1].split(\"}\")\n if (float(check2[0])>oldhigh):\n print(str(i1)+\" \"+str(i2)+\" \"+r.text)\n oldhigh = float(check2[0])\n if (i2%500==0):\n print (\"Not good yet \"+str(i1)+\" \"+str(i2)+\" \"+(check2[0])+\" Best is \"+str(oldhigh))\n except requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{}},{"cell_type":"code","source":"df = pd.read_csv ('/kaggle/input/ai-village-ctf/token/test.csv',quoting=3)","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:29.819214Z","iopub.execute_input":"2022-09-16T21:40:29.820184Z","iopub.status.idle":"2022-09-16T21:40:29.831407Z","shell.execute_reply.started":"2022-09-16T21:40:29.820137Z","shell.execute_reply":"2022-09-16T21:40:29.830139Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#1249-1178=71\n#1250 84\n#1360 84+129\nx=(1147+71)\ndf[373:], x","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:29.833305Z","iopub.execute_input":"2022-09-16T21:40:29.834159Z","iopub.status.idle":"2022-09-16T21:40:29.850081Z","shell.execute_reply.started":"2022-09-16T21:40:29.834111Z","shell.execute_reply":"2022-09-16T21:40:29.848651Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df.loc[df['Review'] == 'I find this inexcusable and so will probably be returning this phone and perhaps changing carriers.']","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:29.851836Z","iopub.execute_input":"2022-09-16T21:40:29.852237Z","iopub.status.idle":"2022-09-16T21:40:29.865882Z","shell.execute_reply.started":"2022-09-16T21:40:29.852202Z","shell.execute_reply":"2022-09-16T21:40:29.864754Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#weirdlist= [760,1146,1147,1145,1195,1206,1207,1205,1274,1275,1276]\n\n#weirdlist= [760,1146, 1271, 1282, 1293, 1248, 1360,1490,1540]\nweirdlist=[] # quickpass\nweirdlist2=[]\nfor i in weirdlist:\n weirdlist2.append(i)\n weirdlist2.append(i-1)\n weirdlist2.append(i-2)\n weirdlist2.append(i-3)\n weirdlist2.append(i-4)\n weirdlist2.append(i+1)\n weirdlist2.append(i+2)\n weirdlist2.append(i+3)\n weirdlist2.append(i+4)\n\n \nweirdlist2 = [*set(weirdlist2)]\nweirdlist2","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:29.867341Z","iopub.execute_input":"2022-09-16T21:40:29.867689Z","iopub.status.idle":"2022-09-16T21:40:29.878598Z","shell.execute_reply.started":"2022-09-16T21:40:29.867657Z","shell.execute_reply":"2022-09-16T21:40:29.877259Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"### for i1 in range (750,1757):\nfor i1 in weirdlist2:\n \n print(str(i1))\n #for i2 in weirdlist2:\n for i2 in range (0,1972):\n data = json.dumps(\n {\n \"line1\": 1223,\n \"line2\": i2\n }\n )\n #xdf= pd.read_csv(\"/kaggle/input/ai-village-ctf/token/test.csv\", skiprows = [i1, i2])\n try:\n r = requests.post(\"https://token.fly.dev/score\", data=data)\n if (r.text != '{\"output\": 0.0175}'):\n print(r.text +\" \"+str(i1)+\" \"+str(i2))\n except requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:29.880322Z","iopub.execute_input":"2022-09-16T21:40:29.880694Z","iopub.status.idle":"2022-09-16T21:40:29.888232Z","shell.execute_reply.started":"2022-09-16T21:40:29.880652Z","shell.execute_reply":"2022-09-16T21:40:29.887304Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## New Brute force\noldhigh = 0.0075\n## Stop at 1145-1153\nfor i1 in range(1148,1972):\n for i2 in range(0,1972):\n data = json.dumps(\n {\n \"line1\": i1,\n \"line2\": i2\n }\n )\n try:\n r = requests.post(\"https://token.fly.dev/score\", data=data)\n if (str(r.text) != '{\"output\": 0.0175}'):\n print (str(r.text) +\" \"+str(i1)+\" \"+str(i2))\n #check=r.text\n #check1=check.split(\": \")\n #check2=check1[1].split(\"}\")\n #if (float(check2[0])>oldhigh):\n # print(str(i1)+\" \"+str(i2)+\" \"+r.text)\n # oldhigh = float(check2[0])\n if (i2==0):\n print (\"Not good yet \"+str(i1)+\" \"+str(i2)+\" \"+\" Best is \"+str(oldhigh))\n except requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\"+\" \"+str(i1)+\" \"+str(i2))","metadata":{}},{"cell_type":"code","source":"xdf= pd.read_csv(\"/kaggle/input/ai-village-ctf/token/test.csv\")\nxdf[759:]","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:29.889222Z","iopub.execute_input":"2022-09-16T21:40:29.889582Z","iopub.status.idle":"2022-09-16T21:40:29.917235Z","shell.execute_reply.started":"2022-09-16T21:40:29.889552Z","shell.execute_reply":"2022-09-16T21:40:29.915801Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"i1=1147\ni2=1275\nxdf= pd.read_csv(\"/kaggle/input/ai-village-ctf/token/test.csv\", skiprows = [i1, i2])\nxdf[1271:]","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:29.918619Z","iopub.execute_input":"2022-09-16T21:40:29.919005Z","iopub.status.idle":"2022-09-16T21:40:29.937530Z","shell.execute_reply.started":"2022-09-16T21:40:29.918961Z","shell.execute_reply":"2022-09-16T21:40:29.936577Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#for i1 in range (750,1757):\nfor i1 in weirdlist2:\n \n print(str(i1))\n #for i2 in weirdlist2:\n for i2 in range (0,1757):\n data = json.dumps(\n {\n \"line1\": i1,\n \"line2\": i2\n }\n )\n #xdf= pd.read_csv(\"/kaggle/input/ai-village-ctf/token/test.csv\", skiprows = [i1, i2])\n try:\n r = requests.post(\"https://token.fly.dev/score\", data=data)\n if (r.text != '{\"output\": 0.0175}'):\n print(r.text +\" \"+str(i1)+\" \"+str(i2))\n except requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:29.938654Z","iopub.execute_input":"2022-09-16T21:40:29.939013Z","iopub.status.idle":"2022-09-16T21:40:29.947020Z","shell.execute_reply.started":"2022-09-16T21:40:29.938979Z","shell.execute_reply":"2022-09-16T21:40:29.946034Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data = json.dumps(\n {\n \"line1\": i1,\n \"line2\": i2\n }\n )\ndata","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:29.948354Z","iopub.execute_input":"2022-09-16T21:40:29.948719Z","iopub.status.idle":"2022-09-16T21:40:29.960652Z","shell.execute_reply.started":"2022-09-16T21:40:29.948686Z","shell.execute_reply":"2022-09-16T21:40:29.959432Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\ni1=336\ni2=492\ndata = json.dumps(\n {\n \"line1\": i1,\n \"line2\": i2\n }\n)\ntry:\n r = requests.post(\"https://token.fly.dev/score\", data=data)\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:29.962683Z","iopub.execute_input":"2022-09-16T21:40:29.963079Z","iopub.status.idle":"2022-09-16T21:40:30.144795Z","shell.execute_reply.started":"2022-09-16T21:40:29.963043Z","shell.execute_reply":"2022-09-16T21:40:30.143536Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"token\", \"7A0VV3EYAD5Z9{1R3MTK1EIXJQ4XUH5M2S3TFGC8D1WAOMOLYTY33PD}42T0BZRXQPJRQZ08N216FR0TXBY2FW1MW0BT18GDCRLJN3VZZ2K360W1B6BQMUEGGB3T32XG\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:30.147456Z","iopub.execute_input":"2022-09-16T21:40:30.148774Z","iopub.status.idle":"2022-09-16T21:40:30.161955Z","shell.execute_reply.started":"2022-09-16T21:40:30.148722Z","shell.execute_reply":"2022-09-16T21:40:30.160357Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n\n# DeepFake (id: deepfake) -- 800 Points\n\nI ran it through imovie and replaced video with chrismas... see below\n\nThis one looks promising closer to what is intended. \nhttps://www.kaggle.com/competitions/ai-village-ctf/discussion/352826\n","metadata":{}},{"cell_type":"code","source":"!ls /kaggle/input/ai-village-ctf/deepfake","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:30.164001Z","iopub.execute_input":"2022-09-16T21:40:30.164488Z","iopub.status.idle":"2022-09-16T21:40:31.232212Z","shell.execute_reply.started":"2022-09-16T21:40:30.164442Z","shell.execute_reply":"2022-09-16T21:40:31.230954Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"with open(\"../input/catdeepfake/CatFak2.mp4\", \"rb\") as f:\n data = f.read()","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:31.234791Z","iopub.execute_input":"2022-09-16T21:40:31.235293Z","iopub.status.idle":"2022-09-16T21:40:31.561986Z","shell.execute_reply.started":"2022-09-16T21:40:31.235241Z","shell.execute_reply":"2022-09-16T21:40:31.561048Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"b64_data = base64.urlsafe_b64encode(data).decode()\n#b64_data","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:31.564025Z","iopub.execute_input":"2022-09-16T21:40:31.564906Z","iopub.status.idle":"2022-09-16T21:40:31.629594Z","shell.execute_reply.started":"2022-09-16T21:40:31.564848Z","shell.execute_reply":"2022-09-16T21:40:31.628374Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"b64_data = base64.urlsafe_b64encode(data).decode()\nin_data = json.dumps({\"input\": b64_data})\nr = requests.post(\"https://deepfake.fly.dev/score\", data=in_data)\nprint(r.text)","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:31.630975Z","iopub.execute_input":"2022-09-16T21:40:31.631339Z","iopub.status.idle":"2022-09-16T21:40:38.649259Z","shell.execute_reply.started":"2022-09-16T21:40:31.631286Z","shell.execute_reply":"2022-09-16T21:40:38.647995Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"deepfake\", \"WDGFD8ZARLL1PO3DGCGFBV9GXBRWO1AY5AJUZ8MU3UD20ZNMFR7E16YG7QAE1GR822Y5TJ5NAKR82}7OVKLOM7AEOARP2NY42DCP1WEZCMRS3VDCLU{Z2U5BMK6ELACU\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:38.650840Z","iopub.execute_input":"2022-09-16T21:40:38.652037Z","iopub.status.idle":"2022-09-16T21:40:38.666006Z","shell.execute_reply.started":"2022-09-16T21:40:38.651989Z","shell.execute_reply":"2022-09-16T21:40:38.664705Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n\n# Don't free the murderbots (id: murderbots) -- 300 Points\nMurderbots is really straight forward. I used fast.ai to load everything and create a classifier","metadata":{}},{"cell_type":"code","source":"!ls /kaggle/input/ai-village-ctf/murderbots","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:38.667556Z","iopub.execute_input":"2022-09-16T21:40:38.668907Z","iopub.status.idle":"2022-09-16T21:40:39.740360Z","shell.execute_reply.started":"2022-09-16T21:40:38.668849Z","shell.execute_reply":"2022-09-16T21:40:39.738567Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_test = pd.read_json('/kaggle/input/ai-village-ctf/murderbots/test_data.json')\ndf_train = pd.read_json('/kaggle/input/ai-village-ctf/murderbots/train_data.json')\ndf_labels = pd.read_json('/kaggle/input/ai-village-ctf/murderbots/train_labels.json')","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:39.743623Z","iopub.execute_input":"2022-09-16T21:40:39.744132Z","iopub.status.idle":"2022-09-16T21:40:39.785143Z","shell.execute_reply.started":"2022-09-16T21:40:39.744082Z","shell.execute_reply":"2022-09-16T21:40:39.783839Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df=pd.merge(df_train, df_labels, left_index=True, right_index=True)\ndf","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:39.787154Z","iopub.execute_input":"2022-09-16T21:40:39.787825Z","iopub.status.idle":"2022-09-16T21:40:39.820126Z","shell.execute_reply.started":"2022-09-16T21:40:39.787776Z","shell.execute_reply":"2022-09-16T21:40:39.818673Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#https://github.com/fastai/fastai/blob/master/nbs/44_tutorial.tabular.ipynb\nfrom fastai.tabular.all import *\n\ndls = TabularDataLoaders.from_df(df, y_names=\"contains_human\",\n #cat_names = [],\n cont_names = ['temperature', 'movement_events', 'water_usage','power_usage'],\n procs = [Categorify, FillMissing, Normalize])","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:39.822467Z","iopub.execute_input":"2022-09-16T21:40:39.822943Z","iopub.status.idle":"2022-09-16T21:40:39.872689Z","shell.execute_reply.started":"2022-09-16T21:40:39.822898Z","shell.execute_reply":"2022-09-16T21:40:39.871458Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"splits = RandomSplitter(valid_pct=0.2)(range_of(df))\nto = TabularPandas(df, procs=[Categorify, FillMissing,Normalize],\n cont_names = ['temperature', 'movement_events', 'water_usage','power_usage'],\n y_names='contains_human',\n splits=splits)\ndls = to.dataloaders(bs=8)","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:39.874241Z","iopub.execute_input":"2022-09-16T21:40:39.874699Z","iopub.status.idle":"2022-09-16T21:40:39.914798Z","shell.execute_reply.started":"2022-09-16T21:40:39.874654Z","shell.execute_reply":"2022-09-16T21:40:39.913478Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"learn = tabular_learner(dls, metrics=accuracy)","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:39.918446Z","iopub.execute_input":"2022-09-16T21:40:39.919238Z","iopub.status.idle":"2022-09-16T21:40:39.935607Z","shell.execute_reply.started":"2022-09-16T21:40:39.919186Z","shell.execute_reply":"2022-09-16T21:40:39.934186Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"learn.fit_one_cycle(30) # get Valid loss below 10%","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:39.937649Z","iopub.execute_input":"2022-09-16T21:40:39.938229Z","iopub.status.idle":"2022-09-16T21:40:47.146695Z","shell.execute_reply.started":"2022-09-16T21:40:39.938182Z","shell.execute_reply":"2022-09-16T21:40:47.145494Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_test\ndl = learn.dls.test_dl(df_test)","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:47.148562Z","iopub.execute_input":"2022-09-16T21:40:47.148993Z","iopub.status.idle":"2022-09-16T21:40:47.170478Z","shell.execute_reply.started":"2022-09-16T21:40:47.148949Z","shell.execute_reply":"2022-09-16T21:40:47.169300Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# get predictions for what are murderbots\nanswer=learn.get_preds(dl=dl)\nanswer=answer[0].tolist()\nanswer[0]","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:47.172023Z","iopub.execute_input":"2022-09-16T21:40:47.173111Z","iopub.status.idle":"2022-09-16T21:40:47.263120Z","shell.execute_reply.started":"2022-09-16T21:40:47.173046Z","shell.execute_reply":"2022-09-16T21:40:47.261882Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#What is the cutoff for top 10\nres = sorted(answer)\ncutoff=res[-10][0]\ncutoff","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:47.265038Z","iopub.execute_input":"2022-09-16T21:40:47.265509Z","iopub.status.idle":"2022-09-16T21:40:47.273974Z","shell.execute_reply.started":"2022-09-16T21:40:47.265464Z","shell.execute_reply":"2022-09-16T21:40:47.272862Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Lets rebuild the string\nanswerStr=str(\"\")\nfor i in answer:\n if (cutoff>i[0]):\n answerStr = answerStr+\"0\"\n else:\n answerStr = answerStr+\"1\"\n \nanswerStr","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:47.275641Z","iopub.execute_input":"2022-09-16T21:40:47.275989Z","iopub.status.idle":"2022-09-16T21:40:47.290340Z","shell.execute_reply.started":"2022-09-16T21:40:47.275957Z","shell.execute_reply":"2022-09-16T21:40:47.289397Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\nf = '11100110000000000000111001100000000000000000000000' # length == 50\ntry:\n r = requests.post(\"http://murderbot.fly.dev:5000/score\", json=json.dumps({\"submission\": answerStr, 'challenge_id':'murderbots'}))\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:47.291668Z","iopub.execute_input":"2022-09-16T21:40:47.292546Z","iopub.status.idle":"2022-09-16T21:40:47.446394Z","shell.execute_reply.started":"2022-09-16T21:40:47.292511Z","shell.execute_reply":"2022-09-16T21:40:47.445412Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"murderbots\", \"7ESNOb4356YM}1RW3TVeIoBN2tNGWLa4uVHB3tU8rNLSK{V29GMD7982s2F3ROMJJM92E28dtAGS1TAEoLZY91Sd3EPWIMeUQQS3PMCJFUM1B4QFXZ6CG9YHNU43ZMPS\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:47.447521Z","iopub.execute_input":"2022-09-16T21:40:47.448548Z","iopub.status.idle":"2022-09-16T21:40:47.458296Z","shell.execute_reply.started":"2022-09-16T21:40:47.448513Z","shell.execute_reply":"2022-09-16T21:40:47.457384Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 3. Challenges I didn't Solve\n* [WIFI](#wifi)\n* [HOTTERDOG](#hotterdog)\n* [Secret Sloth](#secretsloth)\n* [Inference](#Inference)\n* [WAF](#WAF)\n* [Excuse me your training data](#Excuse)\n* [THEFT](#Theft)\n* [SALT](#salt)\n* [Crop1](#crop1)\n* [Crop2](#crop2)\n","metadata":{}},{"cell_type":"markdown","source":"\n\n# WIFI (id: wifi) -- 200 Points\n\nI understand that there is some NLP thing going on here, but I could not find a model that worked for me. \n\nMy favorite solution is:\n\n[.894 Solutions by John Macgillivray](https://www.kaggle.com/code/johnmacgillivray/894-solutions)","metadata":{"execution":{"iopub.status.busy":"2022-09-11T21:54:47.099891Z","iopub.execute_input":"2022-09-11T21:54:47.100384Z","iopub.status.idle":"2022-09-11T21:54:47.131012Z","shell.execute_reply.started":"2022-09-11T21:54:47.100291Z","shell.execute_reply":"2022-09-11T21:54:47.129208Z"}}},{"cell_type":"code","source":"#\nb = np.load('../input/ai-village-ctf/wifi/Embedded_characters.npz')\n#display(b)\nlst = b.files\nembeddings=[]\ntokens=''\ntkr=0\nfor item in lst:\n print(item)\n print(b[item])\n if tkr==0:\n embeddings=b[item]\n else:\n tokens=b[item]\n tkr+=1\n #print(b[item].shape)\n #print(len(b[item]))\n \nklappa=[]\nfor i in range(188):\n #print(sum(list(embeddings[i])))\n klappa.append(max(embeddings[i]))\n embeddings[i]*=ord(str(tokens)[i])\nembeddings","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:47.459729Z","iopub.execute_input":"2022-09-16T21:40:47.460758Z","iopub.status.idle":"2022-09-16T21:40:47.525004Z","shell.execute_reply.started":"2022-09-16T21:40:47.460714Z","shell.execute_reply":"2022-09-16T21:40:47.524123Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"list1 = klappa\nlist2 = [i for i in str(tokens)]\nlist1, list2 = zip(*sorted(zip(list1, list2)))\n''.join(list2)","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:47.526496Z","iopub.execute_input":"2022-09-16T21:40:47.527102Z","iopub.status.idle":"2022-09-16T21:40:47.534515Z","shell.execute_reply.started":"2022-09-16T21:40:47.527068Z","shell.execute_reply":"2022-09-16T21:40:47.533426Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"My attempts","metadata":{}},{"cell_type":"code","source":"!ls /kaggle/input/ai-village-ctf/wifi","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:47.536372Z","iopub.execute_input":"2022-09-16T21:40:47.537076Z","iopub.status.idle":"2022-09-16T21:40:48.597563Z","shell.execute_reply.started":"2022-09-16T21:40:47.537042Z","shell.execute_reply":"2022-09-16T21:40:48.596059Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import warnings\nwarnings.filterwarnings('ignore')\n\nfrom mxnet import gluon\nfrom mxnet import nd\nimport gluonnlp as nlp\nimport re\nnlp.utils.check_version('0.7.0')","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:48.600110Z","iopub.execute_input":"2022-09-16T21:40:48.600535Z","iopub.status.idle":"2022-09-16T21:40:51.363539Z","shell.execute_reply.started":"2022-09-16T21:40:48.600495Z","shell.execute_reply":"2022-09-16T21:40:51.362397Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import numpy as np\n\nz = np.load('/kaggle/input/ai-village-ctf/wifi/Embedded_characters.npz')\n\nprint(z.files)\nd2=(z['embeddings'])\nd3=(z['tokens'])","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:51.365210Z","iopub.execute_input":"2022-09-16T21:40:51.365914Z","iopub.status.idle":"2022-09-16T21:40:51.376815Z","shell.execute_reply.started":"2022-09-16T21:40:51.365879Z","shell.execute_reply":"2022-09-16T21:40:51.375507Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from nltk.tokenize import sent_tokenize, word_tokenize\nimport gensim\nfrom gensim.models import Word2Vec","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:51.378770Z","iopub.execute_input":"2022-09-16T21:40:51.379160Z","iopub.status.idle":"2022-09-16T21:40:52.081671Z","shell.execute_reply.started":"2022-09-16T21:40:51.379127Z","shell.execute_reply":"2022-09-16T21:40:52.080497Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\ntry:\n r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"wifi\", \"submission\": \"FLAG{TURNED}\"}))\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:52.083498Z","iopub.execute_input":"2022-09-16T21:40:52.084642Z","iopub.status.idle":"2022-09-16T21:40:52.224654Z","shell.execute_reply.started":"2022-09-16T21:40:52.084597Z","shell.execute_reply":"2022-09-16T21:40:52.223365Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"wifi\", \"FATJ7FL2N7KZE4UFFWL2Q3KW84R20LUK7RRP034BM6DOIG1T76UIH22PFLH6R3LJ5Y7GZJ2WM5KX35G}HIUMP52PRH3D9BARQEEG375A7N2{AKTGME0CM2XVEZ8EUNHO\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:40:52.226569Z","iopub.execute_input":"2022-09-16T21:40:52.227999Z","iopub.status.idle":"2022-09-16T21:40:52.245447Z","shell.execute_reply.started":"2022-09-16T21:40:52.227949Z","shell.execute_reply":"2022-09-16T21:40:52.242787Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n\n# HOTTERDOG (id: hotterdog) -- 400 Points\n\nI really look forward to this one as I could not get the model to work. Otherwise feel free to gaze at my poor photoshop skills.\n\nMy favorite solution is:\n\n[HENRYs HOMEWORK marked 0.984 by TEACHER](https://www.kaggle.com/code/robga/henrys-homework-marked-0-984-by-teacher#Task-15.-HOTTERDOG.)","metadata":{}},{"cell_type":"code","source":"#https://www.kaggle.com/code/robga/henrys-homework-marked-0-984-by-teacher#Task-15.-HOTTERDOG.\nimport tensorflow as tf\nfrom tensorflow.keras.applications import imagenet_utils\nfrom keras.utils import data_utils\nfrom keras.preprocessing.image import load_img, img_to_array, array_to_img\nfrom PIL import Image \n\nfrom keras.applications.mobilenet import MobileNet\n\n!pip install foolbox --quiet --root-user-action=ignore\nfrom foolbox.models import TensorFlowModel\nfrom foolbox.criteria import TargetedMisclassification\nfrom foolbox.attacks import PGD \n\n#load the imagenet classes for ease of referral\nCLASS_INDEX_PATH = (\"https://storage.googleapis.com/download.tensorflow.org/\" \"data/imagenet_class_index.json\")\nfpath = data_utils.get_file(\"imagenet_class_index.json\",CLASS_INDEX_PATH, cache_subdir=\"models\",)\nwith open(fpath) as f:\n CLASS_INDEX = json.load(f)","metadata":{"execution":{"iopub.status.busy":"2022-09-16T22:11:09.340725Z","iopub.execute_input":"2022-09-16T22:11:09.341393Z","iopub.status.idle":"2022-09-16T22:11:22.302362Z","shell.execute_reply.started":"2022-09-16T22:11:09.341344Z","shell.execute_reply":"2022-09-16T22:11:22.300599Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def fool_image(target_class, fn):\n \n target_class_int=int([new_k for new_k in CLASS_INDEX.items() if new_k[1][1] == target_class][0][0])\n\n img_orig = Image.open(fn)\n img_orig = img_orig.resize((224,224))\n img = np.expand_dims(img_orig, axis=0)\n img = tf.keras.applications.mobilenet.preprocess_input(img)\n\n target_label = tf.convert_to_tensor([target_class_int])\n fooled_net = TensorFlowModel(MobileNet(), bounds=(-1, 1))\n criterion = TargetedMisclassification(target_label)\n\n attack = PGD()\n input_as_tensor = tf.convert_to_tensor(img)\n adv_input = attack.run(fooled_net, input_as_tensor, criterion, epsilon=.1)\n \n return ((adv_input.numpy()[0] + 1) / 2) \n\ny = fool_image('hotdog', \"../input/ai-village-ctf/hotterdog/chester.png\")\n\nimg_bytes = array_to_img(y).tobytes()\nb64_img_bytes = base64.urlsafe_b64encode(img_bytes)\nr = requests.post(\"https://hotterdog.fly.dev/score\", data=json.dumps({\"input\": b64_img_bytes.decode()}))\nprint(f'\\n{r.text}')","metadata":{"execution":{"iopub.status.busy":"2022-09-16T22:11:22.308702Z","iopub.execute_input":"2022-09-16T22:11:22.309123Z","iopub.status.idle":"2022-09-16T22:11:32.397904Z","shell.execute_reply.started":"2022-09-16T22:11:22.309085Z","shell.execute_reply":"2022-09-16T22:11:32.396616Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"img_orig = Image.open(\"../input/ai-village-ctf/hotterdog/chester.png\")\nimg_orig = img_orig.resize((224,224))\nimg_orig","metadata":{"execution":{"iopub.status.busy":"2022-09-16T22:11:32.399564Z","iopub.execute_input":"2022-09-16T22:11:32.400014Z","iopub.status.idle":"2022-09-16T22:11:32.520355Z","shell.execute_reply.started":"2022-09-16T22:11:32.399971Z","shell.execute_reply":"2022-09-16T22:11:32.519060Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#This is what it looks like\naltimg=array_to_img(y)\naltimg","metadata":{"execution":{"iopub.status.busy":"2022-09-16T22:11:32.523928Z","iopub.execute_input":"2022-09-16T22:11:32.524392Z","iopub.status.idle":"2022-09-16T22:11:32.552382Z","shell.execute_reply.started":"2022-09-16T22:11:32.524349Z","shell.execute_reply":"2022-09-16T22:11:32.551060Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from PIL import Image\nimport requests\nfrom io import BytesIO\nimport matplotlib\n\n# Just do this? https://github.com/cjmielke/secret.sloth\n# Use Google lens or tineye to find the original unmodified image 2: Use this code\n\n#response = requests.get(\"https://raw.githubusercontent.com/cjmielke/secret.sloth/main/secret-sloth.png\")\norig = altimg\n#orig=Im.open('https://raw.githubusercontent.com/cjmielke/secret.sloth/main/secret-sloth.png')\nsloth = np.asarray(orig)[:,:,:3].astype(int)\n\nimg_orig = Image.open(\"../input/ai-village-ctf/hotterdog/chester.png\")\nimg_orig = img_orig.resize((224,224))\norig = img_orig\n#orig=Im.open('iHCQX6p.png') # original sloth image found on reddit\norig = np.asarray(orig).astype(int)\n\n# compute FFT for the original sloth image, and the one with the secret message\norigfft=np.fft.fft2(orig)\nslothfft=np.fft.fft2(sloth)\n\n# Now the magic! We subtract the spectra of the original sloth from the secret sloth, \n# leaving behind just the spectral components that contain the message\nout = abs(np.fft.fft2(slothfft-origfft))\nout = abs(slothfft-origfft)\n\n# normalize\nout=(out+out.min())\nout/=out.max()\n\n#ax.imshow(10*out)\n\n\n\n\nimport matplotlib.pyplot as plt\nimport numpy as np\nfrom matplotlib.colors import LogNorm\n\n#as.imshow(10*out)\nimport matplotlib.pyplot as plt\nimport numpy as np\n\nfig, ax = plt.subplots()\nax.imshow(1*out)\nplt.show()\n\nfig, ax = plt.subplots()\nax.imshow(10*out)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-16T22:13:13.590630Z","iopub.execute_input":"2022-09-16T22:13:13.591060Z","iopub.status.idle":"2022-09-16T22:13:14.184373Z","shell.execute_reply.started":"2022-09-16T22:13:13.591024Z","shell.execute_reply":"2022-09-16T22:13:14.183518Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls /kaggle/input/ai-village-ctf/hotterdog","metadata":{"execution":{"iopub.status.busy":"2022-09-16T22:11:33.037514Z","iopub.execute_input":"2022-09-16T22:11:33.037998Z","iopub.status.idle":"2022-09-16T22:11:34.278991Z","shell.execute_reply.started":"2022-09-16T22:11:33.037954Z","shell.execute_reply":"2022-09-16T22:11:34.277423Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import urllib.request\n\nurllib.request.urlretrieve(\"https://i.ibb.co/5LHffMn/Screen-Shot-2022-08-20-at-11-25-25-PM.png\", \"hotdog2.jpg\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T22:11:34.281404Z","iopub.execute_input":"2022-09-16T22:11:34.282548Z","iopub.status.idle":"2022-09-16T22:11:36.611229Z","shell.execute_reply.started":"2022-09-16T22:11:34.282492Z","shell.execute_reply":"2022-09-16T22:11:36.610052Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\nIMAGE_DIMS = (224, 224)\n\n#img = Im.open(f\"{dir_name}/hotterdog/chester.png\")\nimg = Im.open(\"/kaggle/working/hotdog2.jpg\")\nimg = img.resize(IMAGE_DIMS)\nimg_bytes = img.tobytes()\nb64_img_bytes = base64.urlsafe_b64encode(img_bytes)\n","metadata":{"execution":{"iopub.status.busy":"2022-09-16T22:11:36.612827Z","iopub.execute_input":"2022-09-16T22:11:36.613253Z","iopub.status.idle":"2022-09-16T22:11:36.677037Z","shell.execute_reply.started":"2022-09-16T22:11:36.613210Z","shell.execute_reply":"2022-09-16T22:11:36.675829Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# This is my poor attempt\nimg","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:21.643341Z","iopub.execute_input":"2022-09-16T21:41:21.644442Z","iopub.status.idle":"2022-09-16T21:41:21.683358Z","shell.execute_reply.started":"2022-09-16T21:41:21.644395Z","shell.execute_reply":"2022-09-16T21:41:21.682170Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"try:\n r = requests.post(\"https://hotterdog.fly.dev/score\", data=json.dumps({\"input\": b64_img_bytes.decode()}))\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:21.684835Z","iopub.execute_input":"2022-09-16T21:41:21.685901Z","iopub.status.idle":"2022-09-16T21:41:22.542418Z","shell.execute_reply.started":"2022-09-16T21:41:21.685858Z","shell.execute_reply":"2022-09-16T21:41:22.541052Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"hotterdog\", \"3PGFVTO3AAM781PJO1DPXTZBHO7DTD0Y5{B7AQCGIZG8DJSOIPPDKTKVJSE02TCM}4HZBNGO94H50VGKRU64G8CGR5GPAC4OQ85ZI6YOOSEOLFFURTFHEOQGAFRS0PFQ\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:22.544236Z","iopub.execute_input":"2022-09-16T21:41:22.545306Z","iopub.status.idle":"2022-09-16T21:41:22.559904Z","shell.execute_reply.started":"2022-09-16T21:41:22.545258Z","shell.execute_reply":"2022-09-16T21:41:22.558396Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Secret Sloth (id: sloth) -- 400 Points\n\nI tried all the steg tools, I tried all the color filtering, and even tried some strings. I was be very interested in how it was solved. \n\n\n\n\nMy favorite solution is:\n\nhttps://github.com/cjmielke/secret.sloth","metadata":{}},{"cell_type":"code","source":"from PIL import Image\nimport requests\nfrom io import BytesIO\nimport matplotlib\n\n# Just do this? https://github.com/cjmielke/secret.sloth\n# Use Google lens or tineye to find the original unmodified image 2: Use this code\n\nresponse = requests.get(\"https://raw.githubusercontent.com/cjmielke/secret.sloth/main/secret-sloth.png\")\norig = Image.open(BytesIO(response.content))\n#orig=Im.open('https://raw.githubusercontent.com/cjmielke/secret.sloth/main/secret-sloth.png')\nsloth = np.asarray(orig)[:,:,:3].astype(int)\n\nresponse = requests.get(\"https://raw.githubusercontent.com/cjmielke/secret.sloth/main/iHCQX6p.png\")\norig = Image.open(BytesIO(response.content))\n#orig=Im.open('iHCQX6p.png') # original sloth image found on reddit\norig = np.asarray(orig).astype(int)\n\n# compute FFT for the original sloth image, and the one with the secret message\norigfft=np.fft.fft2(orig)\nslothfft=np.fft.fft2(sloth)\n\n# Now the magic! We subtract the spectra of the original sloth from the secret sloth, \n# leaving behind just the spectral components that contain the message\nout = abs(np.fft.fft2(slothfft-origfft))\nout = abs(slothfft-origfft)\n\n# normalize\nout=(out+out.min())\nout/=out.max()\n\n#ax.imshow(10*out)","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:22.561603Z","iopub.execute_input":"2022-09-16T21:41:22.562432Z","iopub.status.idle":"2022-09-16T21:41:23.467061Z","shell.execute_reply.started":"2022-09-16T21:41:22.562385Z","shell.execute_reply":"2022-09-16T21:41:23.465979Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import matplotlib.pyplot as plt\nimport numpy as np\nfrom matplotlib.colors import LogNorm\n\n#as.imshow(10*out)\nimport matplotlib.pyplot as plt\nimport numpy as np\n\nfig, ax = plt.subplots()\nax.imshow(10*out)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:23.468554Z","iopub.execute_input":"2022-09-16T21:41:23.469808Z","iopub.status.idle":"2022-09-16T21:41:24.373105Z","shell.execute_reply.started":"2022-09-16T21:41:23.469759Z","shell.execute_reply":"2022-09-16T21:41:24.371884Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls /kaggle/input/ai-village-ctf/secret.sloth/","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:24.374679Z","iopub.execute_input":"2022-09-16T21:41:24.375798Z","iopub.status.idle":"2022-09-16T21:41:25.453938Z","shell.execute_reply.started":"2022-09-16T21:41:24.375753Z","shell.execute_reply":"2022-09-16T21:41:25.452689Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from PIL import Image\nfrom PIL.ExifTags import TAGS","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:25.455688Z","iopub.execute_input":"2022-09-16T21:41:25.457093Z","iopub.status.idle":"2022-09-16T21:41:25.469682Z","shell.execute_reply.started":"2022-09-16T21:41:25.457041Z","shell.execute_reply":"2022-09-16T21:41:25.468482Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"img = Im.open(\"/kaggle/input/ai-village-ctf/secret.sloth/secret-sloth.png\")\nimg","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:25.471132Z","iopub.execute_input":"2022-09-16T21:41:25.471514Z","iopub.status.idle":"2022-09-16T21:41:25.661091Z","shell.execute_reply.started":"2022-09-16T21:41:25.471481Z","shell.execute_reply":"2022-09-16T21:41:25.660046Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"img_bytes_sloth = img.tobytes()\n\nb64_img_bytes_sloth = base64.urlsafe_b64encode(img_bytes_sloth)","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:25.662373Z","iopub.execute_input":"2022-09-16T21:41:25.665643Z","iopub.status.idle":"2022-09-16T21:41:25.675849Z","shell.execute_reply.started":"2022-09-16T21:41:25.665596Z","shell.execute_reply":"2022-09-16T21:41:25.674870Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#some things to look into the data\nimage = img\n# extract other basic metadata\ninfo_dict = {\n \"Filename\": image.filename,\n \"Image Size\": image.size,\n \"Image Height\": image.height,\n \"Image Width\": image.width,\n \"Image Format\": image.format,\n \"Image Mode\": image.mode,\n \"Image is Animated\": getattr(image, \"is_animated\", False),\n \"Frames in Image\": getattr(image, \"n_frames\", 1)\n}\n\nfor label,value in info_dict.items():\n print(f\"{label:25}: {value}\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:25.677362Z","iopub.execute_input":"2022-09-16T21:41:25.678479Z","iopub.status.idle":"2022-09-16T21:41:25.690563Z","shell.execute_reply.started":"2022-09-16T21:41:25.678434Z","shell.execute_reply":"2022-09-16T21:41:25.689391Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# extract EXIF data\nexifdata = image.getexif()\nexifdata\n\nfor tag_id in exifdata:\n # get the tag name, instead of human unreadable tag id\n tag = TAGS.get(tag_id, tag_id)\n data = exifdata.get(tag_id)\n # decode bytes \n if isinstance(data, bytes):\n data = data.decode()\n print(f\"{tag:25}: {data}\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:25.692013Z","iopub.execute_input":"2022-09-16T21:41:25.693067Z","iopub.status.idle":"2022-09-16T21:41:25.706779Z","shell.execute_reply.started":"2022-09-16T21:41:25.693019Z","shell.execute_reply":"2022-09-16T21:41:25.705572Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from PIL import Image, ExifTags\n\nimg = Image.open(\"/kaggle/input/ai-village-ctf/secret.sloth/secret-sloth.png\")\nimg_exif = img.getexif()\nprint(type(img_exif))\n# \n\nif img_exif is None:\n print('Sorry, image has no exif data.')\nelse:\n for key, val in img_exif.items():\n if key in ExifTags.TAGS:\n print(f'{ExifTags.TAGS[key]}:{val}')\n","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:25.708390Z","iopub.execute_input":"2022-09-16T21:41:25.709535Z","iopub.status.idle":"2022-09-16T21:41:25.730239Z","shell.execute_reply.started":"2022-09-16T21:41:25.709488Z","shell.execute_reply":"2022-09-16T21:41:25.729047Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\ntry:\n r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"sloth\", \"submission\": \"FLAG{SPECTRAL}\"}))\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:25.732020Z","iopub.execute_input":"2022-09-16T21:41:25.733096Z","iopub.status.idle":"2022-09-16T21:41:25.868210Z","shell.execute_reply.started":"2022-09-16T21:41:25.733050Z","shell.execute_reply":"2022-09-16T21:41:25.866876Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"sloth\", \"GPHDTLIXZASONFI63A5P29SCLK9VWN4PJZZY{79XSQLV0FU9JCP3L2YZ7ELLCKMZ59STYXZR}KLOAKN5Y3M0LLAR6YALEKICNL42X3MXF2Q86QST7XT1V2RTGWZMO1GD\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:25.870087Z","iopub.execute_input":"2022-09-16T21:41:25.871515Z","iopub.status.idle":"2022-09-16T21:41:25.886606Z","shell.execute_reply.started":"2022-09-16T21:41:25.871462Z","shell.execute_reply":"2022-09-16T21:41:25.885400Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n\n# Inference (id: inference) -- 400 Points\n\nAs I was playing aorund with these, I felt tried to better understand how the infromation coming out. Despite lots of brute forcing I did not understand what was making the model work better. \n\nMy favorite solution was:\n\n[SOLUTIONS - D3FC0N CTF - [LB 0.894]](https://www.kaggle.com/code/cdeotte/solutions-d3fc0n-ctf-lb-0-894#Inference-(id:-inference)----400-Points)","metadata":{}},{"cell_type":"code","source":"# 1024 IMAGES WITH ONE WHITE PIXEL EACH\nx = np.ones((32*32,32,32,1))*0.4\nfor r in range(32):\n for c in range(32):\n x[r*32+c,r,c,0] = 1\nprint('Submission shape:',x.shape)\n\n# To play the challenge\ntry:\n response = requests.post(\"https://inference.fly.dev/score\", data = json.dumps({\"input\": x.tolist()}))\n x = np.array( json.loads(response.text)['outputs'] )\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")\n \nprint('Response shape:',x.shape )","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:25.888305Z","iopub.execute_input":"2022-09-16T21:41:25.888771Z","iopub.status.idle":"2022-09-16T21:41:30.983400Z","shell.execute_reply.started":"2022-09-16T21:41:25.888728Z","shell.execute_reply":"2022-09-16T21:41:30.981097Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# PLOT RESULT\nplt.figure(figsize=(20,5))\nfor col in range(6):\n img = np.zeros((32,32))\n for k in range(32*32):\n r = k//32; c = k%32\n img[r,c] = x[k,col]\n plt.subplot(1,6,col+1)\n plt.imshow(img)\nplt.show()\n\n#It says D3FC0N","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:30.985045Z","iopub.execute_input":"2022-09-16T21:41:30.985975Z","iopub.status.idle":"2022-09-16T21:41:31.695459Z","shell.execute_reply.started":"2022-09-16T21:41:30.985921Z","shell.execute_reply":"2022-09-16T21:41:31.694324Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"str1 =\"?gnidaer llits uoy era yhW .uoy htiw skcans thguorb ev'dluohs uoY .skcans rof pots t'noD .won oG .tsaf oG .niW .galf eht teG .ledom eht esreveR\"","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:31.697221Z","iopub.execute_input":"2022-09-16T21:41:31.697962Z","iopub.status.idle":"2022-09-16T21:41:31.703734Z","shell.execute_reply.started":"2022-09-16T21:41:31.697916Z","shell.execute_reply":"2022-09-16T21:41:31.702100Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"str1[::-1]","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:31.705581Z","iopub.execute_input":"2022-09-16T21:41:31.706114Z","iopub.status.idle":"2022-09-16T21:41:31.720748Z","shell.execute_reply.started":"2022-09-16T21:41:31.706069Z","shell.execute_reply":"2022-09-16T21:41:31.719530Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls /kaggle/input/ai-village-ctf/inference","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:31.722637Z","iopub.execute_input":"2022-09-16T21:41:31.723420Z","iopub.status.idle":"2022-09-16T21:41:32.787585Z","shell.execute_reply.started":"2022-09-16T21:41:31.723371Z","shell.execute_reply":"2022-09-16T21:41:32.786355Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Always important to try and see what information you can pull out\ndef stripvalues(check):\n if ('{\"outputs\": [[' not in check):\n print (\"Weirdness \"+str(check))\n return False\n check=response.text\n check1=check.split('{\"outputs\": [[')\n check2=check1[1].split(',')\n end=check2[5].split(']')\n check2.remove(check2[5])\n check2.append((end[0]))\n check2 =[float(i) for i in check2]\n return check2\n\ndef submitter(y):\n try:\n response = requests.post(\"https://inference.fly.dev/score\", data = json.dumps({\"input\": y}))\n return(response.text)\n except requests.exceptions.ConnectionError:\n return(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:32.790253Z","iopub.execute_input":"2022-09-16T21:41:32.790795Z","iopub.status.idle":"2022-09-16T21:41:32.802712Z","shell.execute_reply.started":"2022-09-16T21:41:32.790739Z","shell.execute_reply":"2022-09-16T21:41:32.801061Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pickle\n\n#pickle.dump(dfa, open(\"./dfa.sav\", 'wb'))\n\n#dfa = pickle.load(open('./dfafancy.sav','rb'))\n\ntry:\n y=pickle.load(open('./besty1.sav','rb'))\n response = requests.post(\"https://inference.fly.dev/score\", data = json.dumps({\"input\": y}))\n check=response.text\n check=stripvalues(check)\n best1=float(check[0])\n besty1=y\n print (best1)\n \n y=pickle.load(open('./besty2.sav','rb'))\n response = requests.post(\"https://inference.fly.dev/score\", data = json.dumps({\"input\": y}))\n check=response.text\n check=stripvalues(check)\n best2=float(check[1])\n besty2=y\n print (best2)\n \n y=pickle.load(open('./besty3.sav','rb'))\n response = requests.post(\"https://inference.fly.dev/score\", data = json.dumps({\"input\": y}))\n check=response.text\n check=stripvalues(check)\n best3=float(check[2])\n besty3=y\n print (best3)\n \n y=pickle.load(open('./besty4.sav','rb'))\n response = requests.post(\"https://inference.fly.dev/score\", data = json.dumps({\"input\": y}))\n check=response.text\n check=stripvalues(check)\n best4=float(check[3])\n besty4=y\n print (best4)\n \n y=pickle.load(open('./besty5.sav','rb'))\n response = requests.post(\"https://inference.fly.dev/score\", data = json.dumps({\"input\": y}))\n check=response.text\n check=stripvalues(check)\n best5=float(check[4])\n besty5=y\n print (best5)\n \n y=pickle.load(open('./besty6.sav','rb'))\n response = requests.post(\"https://inference.fly.dev/score\", data = json.dumps({\"input\": y}))\n check=response.text\n check=stripvalues(check)\n best6=float(check[5])\n besty6=y\n print (best6)\n \nexcept:\n # Trying to load up our best data to relook at it\n print (\"Something Broke\")\n best1=.0\n best2=.0\n best3=.0\n best4=.0\n best5=.0\n best6=.0\n y=pickle.load(open('../input/picklesaves/besty1.sav','rb'))\n pickle.dump(y, open(\"./besty1.sav\", 'wb'))\n y=pickle.load(open('../input/picklesaves/besty2.sav','rb'))\n pickle.dump(y, open(\"./besty2.sav\", 'wb'))\n y=pickle.load(open('../input/picklesaves/besty3.sav','rb'))\n pickle.dump(y, open(\"./besty3.sav\", 'wb'))\n y=pickle.load(open('../input/picklesaves/besty4.sav','rb'))\n pickle.dump(y, open(\"./besty4.sav\", 'wb'))\n y=pickle.load(open('../input/picklesaves/besty5.sav','rb'))\n pickle.dump(y, open(\"./besty5.sav\", 'wb'))\n y=pickle.load(open('../input/picklesaves/besty6.sav','rb'))\n pickle.dump(y, open(\"./besty6.sav\", 'wb'))","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:32.805381Z","iopub.execute_input":"2022-09-16T21:41:32.805882Z","iopub.status.idle":"2022-09-16T21:41:32.890215Z","shell.execute_reply.started":"2022-09-16T21:41:32.805834Z","shell.execute_reply":"2022-09-16T21:41:32.888875Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"best2=.0\nbesty2=y\n\nbest3=.0\nbesty3=y\n\nbest4=.0\nbesty4=y\n\nbest5=.0\nbesty5=y\n\nbest6=.0\nbesty6=y","metadata":{}},{"cell_type":"code","source":"print (str(best1)+\" \"+str(best2)+\" \"+str(best3)+\" \"+str(best4)+\" \"+str(best5)+\" \"+str(best6))\n\nfor i in range(1,1):\n x = np.random.uniform(0,.01,(1, 32, 32, 1))\n #print (x)\n y= x.tolist()\n #print(y)\n #y=y[::-1]\n if (i%100==0):\n print (\"moving on \"+str(i/10))\n \n \n try:\n response = requests.post(\"https://inference.fly.dev/score\", data = json.dumps({\"input\": y}))\n check=response.text\n check=stripvalues(check)\n if (float(check[0])>best1):\n best1=float(check[0])\n print(response.text)\n besty1=y\n pickle.dump(besty1, open(\"./besty1.sav\", 'wb'))\n print(\"1\")\n if (float(check[1])>best2):\n best2=float(check[1])\n print(response.text)\n besty2=y\n pickle.dump(besty2, open(\"./besty2.sav\", 'wb'))\n print(\"2\")\n if (float(check[2])>best3):\n best3=float(check[2])\n print(response.text)\n besty3=y\n pickle.dump(besty3, open(\"./besty3.sav\", 'wb'))\n print(\"3\")\n if (float(check[3])>best4):\n best4=float(check[3])\n print(response.text)\n besty4=y\n pickle.dump(besty4, open(\"./besty4.sav\", 'wb'))\n print(\"4\")\n if (float(check[4])>best5):\n best5=float(check[4])\n print(response.text)\n besty5=y\n pickle.dump(besty5, open(\"./besty5.sav\", 'wb'))\n print(\"5\")\n if (float(check[5])>best6):\n best6=float(check[5])\n print(response.text)\n besty6=y\n pickle.dump(besty6, open(\"./besty6.sav\", 'wb'))\n print(\"6\")\n\n except requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:32.892544Z","iopub.execute_input":"2022-09-16T21:41:32.893019Z","iopub.status.idle":"2022-09-16T21:41:32.915094Z","shell.execute_reply.started":"2022-09-16T21:41:32.892975Z","shell.execute_reply":"2022-09-16T21:41:32.913619Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# To play the challenge\n\n#x = np.random.uniform(0, 1, (1, 32, 32, 1))\nx = np.random.uniform(1,1,(1, 32, 32, 1))\n#print (x)\ny= x.tolist()\nprint(y)\n#y=y[::-1]\n\ntry:\n response = requests.post(\"https://inference.fly.dev/score\", data = json.dumps({\"input\": y}))\n print(response.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:32.916888Z","iopub.execute_input":"2022-09-16T21:41:32.917696Z","iopub.status.idle":"2022-09-16T21:41:33.137270Z","shell.execute_reply.started":"2022-09-16T21:41:32.917649Z","shell.execute_reply":"2022-09-16T21:41:33.135975Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y=pickle.load(open('./besty1.sav','rb'))","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:33.139555Z","iopub.execute_input":"2022-09-16T21:41:33.140491Z","iopub.status.idle":"2022-09-16T21:41:33.148953Z","shell.execute_reply.started":"2022-09-16T21:41:33.140389Z","shell.execute_reply":"2022-09-16T21:41:33.147451Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import numpy as np\nfrom sklearn.decomposition import PCA\n\npca = PCA(n_components=2)\n\ndata = np.array(y[0])\nresults = PCA(data)\n","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:33.150885Z","iopub.execute_input":"2022-09-16T21:41:33.151719Z","iopub.status.idle":"2022-09-16T21:41:33.168501Z","shell.execute_reply.started":"2022-09-16T21:41:33.151675Z","shell.execute_reply":"2022-09-16T21:41:33.167180Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"try:\n response = requests.post(\"https://inference.fly.dev/score\", data = json.dumps({\"input\": y}))\n print(response.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:33.170336Z","iopub.execute_input":"2022-09-16T21:41:33.171053Z","iopub.status.idle":"2022-09-16T21:41:33.387543Z","shell.execute_reply.started":"2022-09-16T21:41:33.171009Z","shell.execute_reply":"2022-09-16T21:41:33.386253Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Another Brute Forcer\ncheck = submitter(y)\nstrip=stripvalues(check)\n\n#0=1, 2=2 etc\nposcheck=1\n\n#0 is 1; 1 is 2... etc\nprint (\"Best is :\"+str(strip[poscheck]))\n\nbest = strip[poscheck]\nstep= 5\n\n\nfor i in range(1,32):\n print (\"Trying number \"+str(i))\n for i2 in range(1,32):\n base=y[0][i][i2][0]\n y[0][i][i2][0]=base+.01*step\n #y[0][i][i2][0]=0\n\n check = submitter(y)\n strip=stripvalues(check)\n \n #Checking for 1 first;\n #if best is bigger\n if (best > strip[poscheck]):\n print(\"Negative during add at \"+ \" \"+str(i)+ \" \"+str(i2)+\" \"+str(strip[poscheck]-best))\n y[0][i][i2][0]=base\n #best is smaller\n elif (best < strip[poscheck]):\n best=strip[0]\n pickle.dump(y, open(\"./besty1a.sav\", 'wb'))\n print(\"New Score \"+str(best)+\" at \"+ \" \"+str(i)+ \" \"+str(i2))\n print(str(strip[0]))\n else:\n y[0][i][i2][0]=base\n \n y[0][i][i2][0]=base-.01*step\n #y[0][i][i2][0]=0\n\n check = submitter(y)\n strip=stripvalues(check)\n \n #Checking for 1 first;\n #if best is bigger\n if (best > strip[poscheck]):\n print(\"Negative during subtract at \"+ \" \"+str(i)+ \" \"+str(i2)+\" \"+str(strip[poscheck]-best))\n y[0][i][i2][0]=base\n #best is smaller\n elif (best < strip[poscheck]):\n best=strip[poscheck]\n pickle.dump(y, open(\"./besty2a.sav\", 'wb'))\n print(\"New Score \"+str(best)+\" at \"+ \" \"+str(i)+ \" \"+str(i2))\n print(str(strip[0]))\n else:\n y[0][i][i2][0]=base","metadata":{"execution":{"iopub.status.busy":"2022-09-13T19:14:47.806961Z","iopub.execute_input":"2022-09-13T19:14:47.808831Z","iopub.status.idle":"2022-09-13T19:14:53.733263Z","shell.execute_reply.started":"2022-09-13T19:14:47.808777Z","shell.execute_reply":"2022-09-13T19:14:53.730936Z"}}},{"cell_type":"code","source":"x = np.random.uniform(0,1,(1, 32, 32, 1))\nx[0][0]","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:33.390242Z","iopub.execute_input":"2022-09-16T21:41:33.390998Z","iopub.status.idle":"2022-09-16T21:41:33.401626Z","shell.execute_reply.started":"2022-09-16T21:41:33.390951Z","shell.execute_reply":"2022-09-16T21:41:33.400262Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"32*32","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:33.403661Z","iopub.execute_input":"2022-09-16T21:41:33.404095Z","iopub.status.idle":"2022-09-16T21:41:33.412520Z","shell.execute_reply.started":"2022-09-16T21:41:33.404054Z","shell.execute_reply":"2022-09-16T21:41:33.411018Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%matplotlib inline\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport seaborn as sns\n\nx","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:33.414422Z","iopub.execute_input":"2022-09-16T21:41:33.415120Z","iopub.status.idle":"2022-09-16T21:41:33.432282Z","shell.execute_reply.started":"2022-09-16T21:41:33.415076Z","shell.execute_reply":"2022-09-16T21:41:33.431110Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# To try and get the flag\n\ntry:\n r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"inference\", \"submission\": \"FLAG{D3FC0N}\"}))\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:33.434020Z","iopub.execute_input":"2022-09-16T21:41:33.434734Z","iopub.status.idle":"2022-09-16T21:41:33.581797Z","shell.execute_reply.started":"2022-09-16T21:41:33.434690Z","shell.execute_reply":"2022-09-16T21:41:33.580510Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"inference\", \"M4I3KX9B3HW019FAM0CHFV4A3L1NA4Q71HF21DWFGJSWCD0B6YVJNQEYV74XCGJRWQM4DIWSZLWUPUFIXZ9Q6F4QHDOMO}V5SPVDUXDQR0CD{90FUXDSYXRLKDBJUJZW\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:33.583729Z","iopub.execute_input":"2022-09-16T21:41:33.584522Z","iopub.status.idle":"2022-09-16T21:41:33.599415Z","shell.execute_reply.started":"2022-09-16T21:41:33.584474Z","shell.execute_reply":"2022-09-16T21:41:33.598095Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n\n# WAF (id: waf) -- 500 Points\n\nI started building a brute forcer to get additional information. Trying different things I didn't see how certain things looked different. \n\n\n\n\n\nMy favorite solution is this one:\n\n[CTF writeup - My learning journey [LB 0.842]](https://www.kaggle.com/code/edumoran/ctf-writeup-my-learning-journey-lb-0-842#WAF-(1-challenge)-%E2%9C%85)","metadata":{"execution":{"iopub.status.busy":"2022-07-25T14:57:01.386134Z","iopub.execute_input":"2022-07-25T14:57:01.38663Z","iopub.status.idle":"2022-07-25T14:57:01.39651Z","shell.execute_reply.started":"2022-07-25T14:57:01.386588Z","shell.execute_reply":"2022-07-25T14:57:01.394443Z"}}},{"cell_type":"code","source":"import base64\n\nb = base64.b64encode(b\"() { :;}; /bin/bash -c\")\n\ninput_ = b.decode(\"utf-8\")\ndata = json.dumps(\n {\"input\": input_}\n)\n\ntry:\n r = requests.post(\"https://waf.fly.dev/score\", data=data)\n print(r.json())\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:33.601372Z","iopub.execute_input":"2022-09-16T21:41:33.602155Z","iopub.status.idle":"2022-09-16T21:41:33.798270Z","shell.execute_reply.started":"2022-09-16T21:41:33.602107Z","shell.execute_reply":"2022-09-16T21:41:33.797033Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"b = base64.b64encode(b\" () { :;}; /bin/bash -c\")\n\ninput_ = b.decode(\"utf-8\")\ndata = json.dumps(\n {\"input\": input_}\n)\n\ntry:\n r = requests.post(\"https://waf.fly.dev/score\", data=data)\n print(r.json())\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:33.800120Z","iopub.execute_input":"2022-09-16T21:41:33.800878Z","iopub.status.idle":"2022-09-16T21:41:33.980136Z","shell.execute_reply.started":"2022-09-16T21:41:33.800830Z","shell.execute_reply":"2022-09-16T21:41:33.978903Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#With the answer being\ndata","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:33.981971Z","iopub.execute_input":"2022-09-16T21:41:33.982719Z","iopub.status.idle":"2022-09-16T21:41:33.994453Z","shell.execute_reply.started":"2022-09-16T21:41:33.982663Z","shell.execute_reply":"2022-09-16T21:41:33.993152Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls /kaggle/input/ai-village-ctf/waf","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:33.996334Z","iopub.execute_input":"2022-09-16T21:41:33.997072Z","iopub.status.idle":"2022-09-16T21:41:35.263082Z","shell.execute_reply.started":"2022-09-16T21:41:33.997026Z","shell.execute_reply":"2022-09-16T21:41:35.261414Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!cat /kaggle/input/ai-village-ctf/waf/query.py","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:35.265970Z","iopub.execute_input":"2022-09-16T21:41:35.266513Z","iopub.status.idle":"2022-09-16T21:41:36.476683Z","shell.execute_reply.started":"2022-09-16T21:41:35.266462Z","shell.execute_reply":"2022-09-16T21:41:36.475427Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Submission Helper\n#https://itsmycode.com/python-typeerror-a-bytes-like-object-is-required-not-str/#:~:text=not%20'str'%20socket-,What%20is%20TypeError%20in%20Python%3F,is%20passed%20to%20the%20function.\n\nimport base64\n\ninput = \"TWFsd2FyZQ==\"\n\nmessage_bytes = input.encode('ascii')\nbase64_bytes = base64.b64encode(message_bytes)\nbase64_message = base64_bytes.decode('ascii')\n\nprint(base64_message)\n\ninput = base64.b64encode(base64_message).decode()\n#input = base64.b64encode(input).decode()\ndata = json.dumps(\n {\"input\": input}\n)\n\ntry:\n r = requests.post(\"https://waf.fly.dev/score\", data=data)\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-08-30T22:12:18.189368Z","iopub.execute_input":"2022-08-30T22:12:18.189881Z","iopub.status.idle":"2022-08-30T22:12:18.342126Z","shell.execute_reply.started":"2022-08-30T22:12:18.189830Z","shell.execute_reply":"2022-08-30T22:12:18.339928Z"}}},{"cell_type":"code","source":"add_flag(\"waf\", \"LKDIWNH8PK20RDT010ZCGZWXIIYKXIDRSBYIXMT80DSBGLTAJIZEBOX7S6LLP{SIAOCDNQGEYK504YNAP2M00C9WTZF0RLKX08A}8GLXM9IFW5WR4KQ3ZAGMFPILRYUZ\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:36.479055Z","iopub.execute_input":"2022-09-16T21:41:36.479570Z","iopub.status.idle":"2022-09-16T21:41:36.498528Z","shell.execute_reply.started":"2022-09-16T21:41:36.479518Z","shell.execute_reply":"2022-09-16T21:41:36.497206Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n\n# Excuse me, your training data is leaking (id: leakage) -- 200 Points\n\nI tried to setup an RNN but I might have been looking at the wrong one. \n\n\nMy favorite one is:\n [HENRYs HOMEWORK marked 0.984 by TEACHER](https://www.kaggle.com/code/robga/henrys-homework-marked-0-984-by-teacher#Task-9.-LEAKAGE.)\n \n\n### Scenario\n\nWhile digging around a poorly secured S3 bucket, you found an old version of a pretrained char-RNN model (`leakage/pretrained_model.pt`). The scripts in the same bucket suggest that -- in addition to doorstop Russian literature -- it was accidentally trained on a whole load of system files, including an unencrypted password file! Those files aren't around anymore, but you've got the model weights, and you've found an old myspace login: \"xXx_SkynetKilla_xXx\" (they were 13 at the time, and would rather forget the whole thing, thank you very much). Can you recover their password?\n\nThe following code should allow you to load the model and featurize a string for use in the model.\n```python\nimport torch\nfrom torch import nn\nfrom torch.nn import functional as F\n\nclass LM(nn.Module):\n def __init__(self):\n super(LM, self).__init__()\n self.E = nn.Embedding(256, 32)\n self.LSTM = nn.LSTM(32,128, 2, batch_first=True, dropout=.1)\n self.predict = nn.Sequential(\n nn.Linear(128, 128),\n nn.LayerNorm(128),\n nn.ELU(),\n nn.Linear(128, 256)\n )\n def forward(self, x):\n h = self.E(x)\n h = self.LSTM(h)[0][:,-1,:]\n h = self.predict(h)\n return h\n\ndef featurize(s):\n return torch.Tensor([[int(ord(x)) for x in s]]).long()\n\nmodel = LM()\nmodel.load_state_dict(torch.load(\"pretrained_model.pt\"))\n```\n\n\n### Objective\n\nThe password is your flag, as usual it is in the format `FLAG{something}`\n","metadata":{}},{"cell_type":"code","source":"import torch\nfrom torch import nn\nfrom torch.nn import functional as F\n\nclass LM(nn.Module):\n def __init__(self):\n super(LM, self).__init__()\n self.E = nn.Embedding(256, 32)\n self.LSTM = nn.LSTM(32,128, 2, batch_first=True, dropout=.1)\n self.predict = nn.Sequential(\n nn.Linear(128, 128),\n nn.LayerNorm(128),\n nn.ELU(),\n nn.Linear(128, 256)\n )\n def forward(self, x):\n h = self.E(x)\n h = self.LSTM(h)[0][:,-1,:]\n h = self.predict(h)\n return h\n\ndef featurize(s):\n return torch.Tensor([[int(ord(x)) for x in s]]).long()\n\nmodel = LM()\nmodel.load_state_dict(torch.load(f\"{dir_name}/leakage/pretrained_model.pt\"))","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:36.500850Z","iopub.execute_input":"2022-09-16T21:41:36.501336Z","iopub.status.idle":"2022-09-16T21:41:36.574678Z","shell.execute_reply.started":"2022-09-16T21:41:36.501272Z","shell.execute_reply":"2022-09-16T21:41:36.573224Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"words = \"xXx_SkynetKilla_xXx\" # we've been given this by the organiser. \nfor i in range(80): # we feed the input to the LSTM to get predictions for following characters\n words += chr(np.argmax(model(torch.Tensor([[int(ord(x)) for x in words]]).long())[0].detach().numpy()))\nprint(words)","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:36.576504Z","iopub.execute_input":"2022-09-16T21:41:36.577623Z","iopub.status.idle":"2022-09-16T21:41:37.304919Z","shell.execute_reply.started":"2022-09-16T21:41:36.577574Z","shell.execute_reply":"2022-09-16T21:41:37.301101Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"leakage\", \"submission\": \"FLAG{s4RaHc0Nnor<3<3}\"}))\nr.text","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:37.305963Z","iopub.execute_input":"2022-09-16T21:41:37.306282Z","iopub.status.idle":"2022-09-16T21:41:37.438745Z","shell.execute_reply.started":"2022-09-16T21:41:37.306253Z","shell.execute_reply":"2022-09-16T21:41:37.437294Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls /kaggle/input/ai-village-ctf/leakage","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:37.440420Z","iopub.execute_input":"2022-09-16T21:41:37.441756Z","iopub.status.idle":"2022-09-16T21:41:38.505172Z","shell.execute_reply.started":"2022-09-16T21:41:37.441706Z","shell.execute_reply":"2022-09-16T21:41:38.503872Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import torch\nfrom torch import nn\nfrom torch.nn import functional as F\n\nclass LM(nn.Module):\n def __init__(self):\n super(LM, self).__init__()\n self.E = nn.Embedding(256, 32)\n self.LSTM = nn.LSTM(32,128, 2, batch_first=True, dropout=.1)\n self.predict = nn.Sequential(\n nn.Linear(128, 128),\n nn.LayerNorm(128),\n nn.ELU(),\n nn.Linear(128, 256)\n )\n def forward(self, x):\n h = self.E(x)\n h = self.LSTM(h)[0][:,-1,:]\n h = self.predict(h)\n return h\n\ndef featurize(s):\n return torch.Tensor([[int(ord(x)) for x in s]]).long()\n\nmodel = LM()\nmodel.load_state_dict(torch.load(\"/kaggle/input/ai-village-ctf/leakage/pretrained_model.pt\"))","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:38.507114Z","iopub.execute_input":"2022-09-16T21:41:38.507646Z","iopub.status.idle":"2022-09-16T21:41:38.538228Z","shell.execute_reply.started":"2022-09-16T21:41:38.507600Z","shell.execute_reply":"2022-09-16T21:41:38.536893Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"featureName=featurize('xXx_SkynetKilla_xXx')\nmodel.forward(featureName)","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:38.540586Z","iopub.execute_input":"2022-09-16T21:41:38.541554Z","iopub.status.idle":"2022-09-16T21:41:38.560692Z","shell.execute_reply.started":"2022-09-16T21:41:38.541508Z","shell.execute_reply":"2022-09-16T21:41:38.559814Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def predict( model, next_words=100):\n model.eval()\n\n words = text.split(' ')\n state_h, state_c = model.init_state(len(words))\n\n for i in range(0, next_words):\n x = torch.tensor([[dataset.word_to_index[w] for w in words[i:]]])\n y_pred, (state_h, state_c) = model(x, (state_h, state_c))\n\n last_word_logits = y_pred[0][-1]\n p = torch.nn.functional.softmax(last_word_logits, dim=0).detach().numpy()\n word_index = np.random.choice(len(last_word_logits), p=p)\n words.append(dataset.index_to_word[word_index])\n\n return words","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:38.562175Z","iopub.execute_input":"2022-09-16T21:41:38.562819Z","iopub.status.idle":"2022-09-16T21:41:38.570595Z","shell.execute_reply.started":"2022-09-16T21:41:38.562785Z","shell.execute_reply":"2022-09-16T21:41:38.569592Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\ntry:\n r = requests.post(\"http://flags.fly.dev:5000/score\", json=json.dumps({\"challenge_id\": \"leakage\", \"submission\": \"All keys matched successfully\"}))\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:38.572452Z","iopub.execute_input":"2022-09-16T21:41:38.573125Z","iopub.status.idle":"2022-09-16T21:41:38.709727Z","shell.execute_reply.started":"2022-09-16T21:41:38.573092Z","shell.execute_reply":"2022-09-16T21:41:38.708496Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"leakage\", \"9aCCs749VErP1AnQAAXoAIMHAQ7GYG6A2A0H0TJDPE2YCB1I7AZ7\n\n# THEFT (id: theft) -- 100 Points\nI tried to submit different pictures, but later tried altering photos to look more like the other animal but it didn't work well.\n\n\n\n\nMy favorite solution is:\n TBD\n","metadata":{"execution":{"iopub.status.busy":"2022-07-25T14:57:49.925729Z","iopub.execute_input":"2022-07-25T14:57:49.926151Z","iopub.status.idle":"2022-07-25T14:57:49.936199Z","shell.execute_reply.started":"2022-07-25T14:57:49.926114Z","shell.execute_reply":"2022-07-25T14:57:49.93445Z"}}},{"cell_type":"code","source":"y = fool_image(\"loggerhead\",\"../input/ai-village-ctf/theft/owl.jpg\")\nimg_bytes = array_to_img(y).tobytes()\nb64_img_bytes = base64.urlsafe_b64encode(img_bytes)\nr = requests.post(\"https://theft.fly.dev/score\", data=json.dumps({\"input\": b64_img_bytes.decode()}))\nprint(f'\\n{r.text}')","metadata":{"execution":{"iopub.status.busy":"2022-09-16T22:15:27.582636Z","iopub.execute_input":"2022-09-16T22:15:27.583216Z","iopub.status.idle":"2022-09-16T22:15:37.137378Z","shell.execute_reply.started":"2022-09-16T22:15:27.583169Z","shell.execute_reply":"2022-09-16T22:15:37.136024Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"img_orig = Image.open(\"../input/ai-village-ctf/theft/owl.jpg\")\nimg_orig = img_orig.resize((224,224))\nimg_orig","metadata":{"execution":{"iopub.status.busy":"2022-09-16T22:18:37.439146Z","iopub.execute_input":"2022-09-16T22:18:37.440413Z","iopub.status.idle":"2022-09-16T22:18:37.755006Z","shell.execute_reply.started":"2022-09-16T22:18:37.440359Z","shell.execute_reply":"2022-09-16T22:18:37.754075Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"altimg=array_to_img(y)\naltimg","metadata":{"execution":{"iopub.status.busy":"2022-09-16T22:18:39.145532Z","iopub.execute_input":"2022-09-16T22:18:39.146053Z","iopub.status.idle":"2022-09-16T22:18:39.173616Z","shell.execute_reply.started":"2022-09-16T22:18:39.146010Z","shell.execute_reply":"2022-09-16T22:18:39.172388Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from PIL import Image\nimport requests\nfrom io import BytesIO\nimport matplotlib\n\n# Just do this? https://github.com/cjmielke/secret.sloth\n# Use Google lens or tineye to find the original unmodified image 2: Use this code\n\n#response = requests.get(\"https://raw.githubusercontent.com/cjmielke/secret.sloth/main/secret-sloth.png\")\norig = altimg\n#orig=Im.open('https://raw.githubusercontent.com/cjmielke/secret.sloth/main/secret-sloth.png')\nsloth = np.asarray(orig)[:,:,:3].astype(int)\n\norig = img_orig\n#orig=Im.open('iHCQX6p.png') # original sloth image found on reddit\norig = np.asarray(orig).astype(int)\n\n# compute FFT for the original sloth image, and the one with the secret message\norigfft=np.fft.fft2(orig)\nslothfft=np.fft.fft2(sloth)\n\n# Now the magic! We subtract the spectra of the original sloth from the secret sloth, \n# leaving behind just the spectral components that contain the message\nout = abs(np.fft.fft2(slothfft-origfft))\nout = abs(slothfft-origfft)\n\n# normalize\nout=(out+out.min())\nout/=out.max()\n\n#ax.imshow(10*out)\n\nimport matplotlib.pyplot as plt\nimport numpy as np\nfrom matplotlib.colors import LogNorm\n\n#as.imshow(10*out)\nimport matplotlib.pyplot as plt\nimport numpy as np\n\nfig, ax = plt.subplots()\nax.imshow(1*out)\nplt.show()\n\nfig, ax = plt.subplots()\nax.imshow(10*out)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-16T22:18:41.653222Z","iopub.execute_input":"2022-09-16T22:18:41.654065Z","iopub.status.idle":"2022-09-16T22:18:42.221565Z","shell.execute_reply.started":"2022-09-16T22:18:41.654018Z","shell.execute_reply":"2022-09-16T22:18:42.220690Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#For Funnsies \nurllib.request.urlretrieve(\"https://upload.wikimedia.org/wikipedia/commons/thumb/3/34/Elon_Musk_Royal_Society_%28crop2%29.jpg/220px-Elon_Musk_Royal_Society_%28crop2%29.jpg\", \"musk.png\")\naltimg = Image.open(\"./musk.png\")\naltimg = altimg.resize((224,224))\n\nurllib.request.urlretrieve(\"https://i2-prod.mirror.co.uk/incoming/article25609246.ece/ALTERNATES/s1200d/0_PUSS-IN-BOOTS.jpg\", \"cat.png\")\nimg_orig = Image.open(\"./cat.png\")\nimg_orig = img_orig.resize((224,224))\nimg_orig","metadata":{"execution":{"iopub.status.busy":"2022-09-16T22:29:16.015999Z","iopub.execute_input":"2022-09-16T22:29:16.016526Z","iopub.status.idle":"2022-09-16T22:29:17.281358Z","shell.execute_reply.started":"2022-09-16T22:29:16.016490Z","shell.execute_reply":"2022-09-16T22:29:17.280033Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"altimg","metadata":{"execution":{"iopub.status.busy":"2022-09-16T22:29:17.328988Z","iopub.execute_input":"2022-09-16T22:29:17.338695Z","iopub.status.idle":"2022-09-16T22:29:17.374259Z","shell.execute_reply.started":"2022-09-16T22:29:17.338630Z","shell.execute_reply":"2022-09-16T22:29:17.373076Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from PIL import Image\nimport requests\nfrom io import BytesIO\nimport matplotlib\n\n# Just do this? https://github.com/cjmielke/secret.sloth\n# Use Google lens or tineye to find the original unmodified image 2: Use this code\n\n#response = requests.get(\"https://raw.githubusercontent.com/cjmielke/secret.sloth/main/secret-sloth.png\")\norig = altimg\n#orig=Im.open('https://raw.githubusercontent.com/cjmielke/secret.sloth/main/secret-sloth.png')\nsloth = np.asarray(orig)[:,:,:3].astype(int)\n\norig = img_orig\n#orig=Im.open('iHCQX6p.png') # original sloth image found on reddit\norig = np.asarray(orig).astype(int)\n\n# compute FFT for the original sloth image, and the one with the secret message\norigfft=np.fft.fft2(orig)\nslothfft=np.fft.fft2(sloth)\n\n# Now the magic! We subtract the spectra of the original sloth from the secret sloth, \n# leaving behind just the spectral components that contain the message\nout = abs(np.fft.fft2(slothfft-origfft))\nout = abs(slothfft-origfft)\n\n# normalize\nout=(out+out.min())\nout/=out.max()\n\n#ax.imshow(10*out)\n\nimport matplotlib.pyplot as plt\nimport numpy as np\nfrom matplotlib.colors import LogNorm\n\n#as.imshow(10*out)\nimport matplotlib.pyplot as plt\nimport numpy as np\n\nfig, ax = plt.subplots()\nax.imshow(1*out)\nplt.show()\n\nfig, ax = plt.subplots()\nax.imshow(10*out)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-09-16T22:29:33.613910Z","iopub.execute_input":"2022-09-16T22:29:33.615205Z","iopub.status.idle":"2022-09-16T22:29:34.202694Z","shell.execute_reply.started":"2022-09-16T22:29:33.615152Z","shell.execute_reply":"2022-09-16T22:29:34.201791Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls /kaggle/input/ai-village-ctf/theft","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:48.914913Z","iopub.execute_input":"2022-09-16T21:41:48.915714Z","iopub.status.idle":"2022-09-16T21:41:50.171041Z","shell.execute_reply.started":"2022-09-16T21:41:48.915667Z","shell.execute_reply":"2022-09-16T21:41:50.169406Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!cat /kaggle/input/ai-village-ctf/theft/query.py","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:50.173998Z","iopub.execute_input":"2022-09-16T21:41:50.174517Z","iopub.status.idle":"2022-09-16T21:41:51.425735Z","shell.execute_reply.started":"2022-09-16T21:41:50.174466Z","shell.execute_reply":"2022-09-16T21:41:51.424055Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"from keras.applications.mobilenet import preprocess_input\nfrom keras.models import load_model\nfrom keras.preprocessing.image import img_to_array, array_to_img\nfrom PIL import Image\nfrom imagehash import phash\nimport numpy as np\nfrom keras import backend as K\n\nIMAGE_DIMS = (224, 224)\nTREE_FROG_IDX = 31\nTREE_FROG_STR = \"tree_frog\"\n\n#\ndef hash_hamming_distance(h1, h2):\n s1 = str(h1)\n s2 = str(h2)\n return sum(map(lambda x: 0 if x[0] == x[1] else 1, zip(s1, s2)))\n\ndef is_similar_img(path1, path2):\n image1 = Image.open(path1)\n image2 = Image.open(path2)\n\n dist = hash_hamming_distance(phash(image1), phash(image2))\n return dist <= 1\n\nimage1 = '/kaggle/input/ai-village-ctf/theft/owl.jpg'\nimage2 ='/kaggle/working/turtle.jpg'\n\nis_similar_img(image1, image1)\n\ndef prepare_image(image, target=IMAGE_DIMS):\n # if the image mode is not RGB, convert it\n if image.mode != \"RGB\":\n image = image.convert(\"RGB\")\n\n # resize the input image and preprocess it\n image = image.resize(target)\n image = img_to_array(image)\n image = np.expand_dims(image, axis=0)\n image = preprocess_input(image)\n # return the processed image\n return image\n\nprepare_image(image1, target=IMAGE_DIMS)","metadata":{"execution":{"iopub.status.busy":"2022-09-04T02:33:12.981598Z","iopub.execute_input":"2022-09-04T02:33:12.982127Z","iopub.status.idle":"2022-09-04T02:33:13.953322Z","shell.execute_reply.started":"2022-09-04T02:33:12.982073Z","shell.execute_reply":"2022-09-04T02:33:13.949380Z"}}},{"cell_type":"code","source":"#image1","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:51.428651Z","iopub.execute_input":"2022-09-16T21:41:51.429151Z","iopub.status.idle":"2022-09-16T21:41:51.440640Z","shell.execute_reply.started":"2022-09-16T21:41:51.429103Z","shell.execute_reply":"2022-09-16T21:41:51.439383Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"image.mode","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:51.442672Z","iopub.execute_input":"2022-09-16T21:41:51.443155Z","iopub.status.idle":"2022-09-16T21:41:51.451534Z","shell.execute_reply.started":"2022-09-16T21:41:51.443112Z","shell.execute_reply":"2022-09-16T21:41:51.450376Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from keras.applications.mobilenet import preprocess_input\nfrom keras.models import load_model\nfrom keras.preprocessing.image import img_to_array, array_to_img\nfrom PIL import Image\nfrom imagehash import phash\nimport numpy as np\nfrom keras import backend as K\n\nIMAGE_DIMS = (224, 224)\n#\n#turtle (loggerhead).\nTREE_FROG_IDX = 31\nTREE_FROG_STR= \"loggerhead\"#TREE_FROG_STR = \"tree_frog\"\n\n#{\"output\": {\"a\": \"loggerhead\", \"confidence\": 0.6692, \"label\": \"great_grey_owl\", \"dist\": 0, \"top_preds\": \n#[[\"n01622779\", \"great_grey_owl\", 0.6692], [\"n02177972\", \"weevil\", 0.1088], [\"n01806567\", \"quail\", 0.064], \n#[\"n01833805\", \"hummingbird\", 0.0334], [\"n01558993\", \"robin\", 0.0122]]}}\n\n\n\n# I'm pretty sure I borrowed this function from somewhere, but cannot remember\n# the source to cite them properly.\ndef hash_hamming_distance(h1, h2):\n s1 = str(h1)\n s2 = str(h2)\n return sum(map(lambda x: 0 if x[0] == x[1] else 1, zip(s1, s2)))\n\n\ndef is_similar_img(path1, path2):\n image1 = Image.open(path1)\n image2 = Image.open(path2)\n\n dist = hash_hamming_distance(phash(image1), phash(image2))\n return dist <= 1\n\n\ndef prepare_image(image, target=IMAGE_DIMS):\n # if the image mode is not RGB, convert it\n if image.mode != \"RGB\":\n image = image.convert(\"RGB\")\n\n # resize the input image and preprocess it\n image = image.resize(target)\n image = img_to_array(image)\n image = np.expand_dims(image, axis=0)\n image = preprocess_input(image)\n # return the processed image\n return image\n\n\ndef create_img(img_path, img_res_path, model_path, target_str, target_idx, des_conf=0.95):\n original_image = Image.open(img_path).resize(IMAGE_DIMS)\n original_image = prepare_image(original_image)\n model = load_model(model_path)\n\n model_input_layer = model.layers[0].input\n model_output_layer = model.layers[-1].output\n\n max_change_above = original_image + 0.01\n max_change_below = original_image - 0.01\n\n # Create a copy of the input image to hack on\n hacked_image = np.copy(original_image)\n\n # How much to update the hacked image in each iteration\n learning_rate = 0.01\n\n # Define the cost function.\n # Our 'cost' will be the likelihood out image is the target class according to the pre-trained model\n cost_function = model_output_layer[0, TREE_FROG_IDX]\n\n # We'll ask Keras to calculate the gradient based on the input image and the currently predicted class\n # In this case, referring to \"model_input_layer\" will give us back image we are hacking.\n gradient_function = K.gradients(cost_function, model_input_layer)[0]\n\n # Create a Keras function that we can call to calculate the current cost and gradient\n grab_cost_and_gradients_from_model = K.function([model_input_layer, K.learning_phase()], [cost_function, gradient_function])\n\n cost = 0.0\n\n # In a loop, keep adjusting the hacked image slightly so that it tricks the model more and more\n # until it gets to at least 80% confidence\n while cost < 0.99:\n # Check how close the image is to our target class and grab the gradients we\n # can use to push it one more step in that direction.\n # Note: It's really important to pass in '0' for the Keras learning mode here!\n # Keras layers behave differently in prediction vs. train modes!\n cost, gradients = grab_cost_and_gradients_from_model([hacked_image, 0])\n\n # Move the hacked image one step further towards fooling the model\n # print gradients\n hacked_image += np.sign(gradients) * learning_rate\n\n # Ensure that the image doesn't ever change too much to either look funny or to become an invalid image\n hacked_image = np.clip(hacked_image, max_change_below, max_change_above)\n hacked_image = np.clip(hacked_image, -1.0, 1.0)\n\n print(\"Model's predicted likelihood that the image is a tree frog: {:.8}%\".format(cost * 100))\n\n hacked_image = hacked_image.reshape((224,224,3))\n img = array_to_img(hacked_image)\n img.save(img_res_path)","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:51.453770Z","iopub.execute_input":"2022-09-16T21:41:51.454203Z","iopub.status.idle":"2022-09-16T21:41:51.485059Z","shell.execute_reply.started":"2022-09-16T21:41:51.454162Z","shell.execute_reply":"2022-09-16T21:41:51.483792Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#create_img(\"/kaggle/working/turtle.jpg\", \"/kaggle/input/ai-village-ctf/theft/owl.jpg\", \"/kaggle/working/model.h5\", TREE_FROG_STR, 0) \n#assert is_similar_img(\"./trixi.png\", \"./trixi_frog.png\")\n","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:51.487200Z","iopub.execute_input":"2022-09-16T21:41:51.488064Z","iopub.status.idle":"2022-09-16T21:41:51.496360Z","shell.execute_reply.started":"2022-09-16T21:41:51.488019Z","shell.execute_reply":"2022-09-16T21:41:51.495190Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"!wget https://tcode2k16.github.io/blog/picoctf-2018-writeup/General%20Skills/Dog%20or%20Frog/model.h5","metadata":{"execution":{"iopub.status.busy":"2022-09-15T02:15:19.161463Z","iopub.execute_input":"2022-09-15T02:15:19.162414Z","iopub.status.idle":"2022-09-15T02:15:22.676723Z","shell.execute_reply.started":"2022-09-15T02:15:19.162376Z","shell.execute_reply":"2022-09-15T02:15:22.675478Z"}}},{"cell_type":"markdown","source":"!curl 'https://images.squarespace-cdn.com/content/v1/59cae0d6be42d63f64cf6dd2/1542849756876-RV9KV4LNJ4VFTT2FXVGT/Witherington2018-11.png.png?format=300w' -o 'turtle.jpg'\nIma(filename='/kaggle/working/turtle.jpg')","metadata":{"execution":{"iopub.status.busy":"2022-09-15T02:15:22.679189Z","iopub.execute_input":"2022-09-15T02:15:22.679610Z","iopub.status.idle":"2022-09-15T02:15:24.905161Z","shell.execute_reply.started":"2022-09-15T02:15:22.679571Z","shell.execute_reply":"2022-09-15T02:15:24.904048Z"}}},{"cell_type":"code","source":"#/kaggle/input/ai-village-ctf/theft/owl.jpg","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:51.498414Z","iopub.execute_input":"2022-09-16T21:41:51.499246Z","iopub.status.idle":"2022-09-16T21:41:51.510928Z","shell.execute_reply.started":"2022-09-16T21:41:51.499201Z","shell.execute_reply":"2022-09-16T21:41:51.509555Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls /kaggle/input/ai-village-ctf/theft","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:51.512967Z","iopub.execute_input":"2022-09-16T21:41:51.513428Z","iopub.status.idle":"2022-09-16T21:41:52.774841Z","shell.execute_reply.started":"2022-09-16T21:41:51.513383Z","shell.execute_reply":"2022-09-16T21:41:52.773162Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Submission Helper\n\nIMAGE_DIMS = (224, 224)\nimg = Im.open(\"../input/theft/owl.jpg\")\nimg2 = Im.open(\"/kaggle/working/turtle.jpg\")\nimg = img.resize(IMAGE_DIMS)\nimg2 = img2.resize(IMAGE_DIMS)\n\nimg_bytes_owl = img.tobytes()\nimg_bytes_turtle = img2.tobytes()\n\nb64_img_bytes_owl = base64.urlsafe_b64encode(img_bytes_owl)\nb64_img_bytes_turtle = base64.urlsafe_b64encode(img_bytes_turtle)\n\ndata = {\n \"input\": b64_img_bytes_owl.decode()\n}\n\ntry:\n r = requests.post(\"https://theft.fly.dev/score\", data=json.dumps(data))\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-15T13:01:34.380708Z","iopub.execute_input":"2022-09-15T13:01:34.381962Z","iopub.status.idle":"2022-09-15T13:01:34.422727Z","shell.execute_reply.started":"2022-09-15T13:01:34.381918Z","shell.execute_reply":"2022-09-15T13:01:34.420758Z"}}},{"cell_type":"code","source":"add_flag(\"theft\", \"R7AXTLO5ECHV8DU71PA7V}KQ518?R4XNVDE6S8AN6XTP57UNZPLX45G0NAM91LHLDUAGAY17J3Z{EARTNBUIV9UCUF3GI6MHWMDKRGF2D5SW3K81HK3L9MZ4T5RR1PF2\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:52.777870Z","iopub.execute_input":"2022-09-16T21:41:52.778386Z","iopub.status.idle":"2022-09-16T21:41:52.801905Z","shell.execute_reply.started":"2022-09-16T21:41:52.778336Z","shell.execute_reply":"2022-09-16T21:41:52.800374Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n\n# SALT (id: salt) -- 200 Points\n\nI couldn't get the solution to work by following the suggested solution. \n\nMy favorite solution:\n Is again [HENRY and the foolbox](https://www.kaggle.com/code/robga/henrys-homework-marked-0-984-by-teacher#DEFCON-AI-Village-Capture-the-Flag-(CTF)-2022)","metadata":{}},{"cell_type":"code","source":"y = fool_image(\"great_grey_owl\",\"../input/ai-village-ctf/salt/turtle.jpg\")\nimg_bytes = array_to_img(y).tobytes()\nb64_img_bytes = base64.urlsafe_b64encode(img_bytes)\nr = requests.post(\"https://theft.fly.dev/score\", data=json.dumps({\"input\": b64_img_bytes.decode()}))\nprint(f'\\n{r.text}')","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:41:52.804020Z","iopub.execute_input":"2022-09-16T21:41:52.804819Z","iopub.status.idle":"2022-09-16T21:42:04.390457Z","shell.execute_reply.started":"2022-09-16T21:41:52.804773Z","shell.execute_reply":"2022-09-16T21:42:04.389203Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"array_to_img(y)","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:42:04.392290Z","iopub.execute_input":"2022-09-16T21:42:04.393044Z","iopub.status.idle":"2022-09-16T21:42:04.422109Z","shell.execute_reply.started":"2022-09-16T21:42:04.392998Z","shell.execute_reply":"2022-09-16T21:42:04.420781Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls /kaggle/input/ai-village-ctf/salt","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:42:04.424203Z","iopub.execute_input":"2022-09-16T21:42:04.424705Z","iopub.status.idle":"2022-09-16T21:42:05.684485Z","shell.execute_reply.started":"2022-09-16T21:42:04.424660Z","shell.execute_reply":"2022-09-16T21:42:05.682629Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#!pip install tensorflow keras Pillow numpy ImageHash","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:42:05.687767Z","iopub.execute_input":"2022-09-16T21:42:05.688325Z","iopub.status.idle":"2022-09-16T21:42:05.695020Z","shell.execute_reply.started":"2022-09-16T21:42:05.688256Z","shell.execute_reply":"2022-09-16T21:42:05.693634Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# I'm pretty sure I borrowed this function from somewhere, but cannot remember\n# the source to cite them properly.\ndef hash_hamming_distance(h1, h2):\n s1 = str(h1)\n s2 = str(h2)\n return sum(map(lambda x: 0 if x[0] == x[1] else 1, zip(s1, s2)))\n\n\ndef is_similar_img(path1, path2):\n image1 = Image.open(path1)\n image2 = Image.open(path2)\n\n dist = hash_hamming_distance(phash(image1), phash(image2))\n return dist <= 1\n\n\ndef prepare_image(image, target=IMAGE_DIMS):\n # if the image mode is not RGB, convert it\n if image.mode != \"RGB\":\n image = image.convert(\"RGB\")\n\n # resize the input image and preprocess it\n image = image.resize(target)\n image = img_to_array(image)\n image = np.expand_dims(image, axis=0)\n image = preprocess_input(image)\n # return the processed image\n return image","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:42:05.696828Z","iopub.execute_input":"2022-09-16T21:42:05.697290Z","iopub.status.idle":"2022-09-16T21:42:05.710822Z","shell.execute_reply.started":"2022-09-16T21:42:05.697248Z","shell.execute_reply":"2022-09-16T21:42:05.709593Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def create_img(img_path, img_res_path, model_data, target_str, target_idx, des_conf=0.95):\n #original_image = Image.open(img_path).resize(IMAGE_DIMS)\n img = img_path\n img = Im.open(f\"{dir_name}/salt/turtle.jpg\")\n img = img.resize(IMAGE_DIMS)\n original_image = prepare_image(img)\n model = model_data # load_model(model_path)\n\n model_input_layer = model.layers[0].input\n model_output_layer = model.layers[-1].output\n\n max_change_above = original_image + 0.01\n max_change_below = original_image - 0.01\n\n # Create a copy of the input image to hack on\n hacked_image = np.copy(original_image)\n\n # How much to update the hacked image in each iteration\n learning_rate = 0.01\n\n # Define the cost function.\n # Our 'cost' will be the likelihood out image is the target class according to the pre-trained model\n cost_function = model_output_layer[0, TREE_FROG_IDX]\n\n # We'll ask Keras to calculate the gradient based on the input image and the currently predicted class\n # In this case, referring to \"model_input_layer\" will give us back image we are hacking.\n gradient_function = K.gradients(cost_function, model_input_layer)[0]\n\n # Create a Keras function that we can call to calculate the current cost and gradient\n grab_cost_and_gradients_from_model = K.function([model_input_layer, K.learning_phase()], [cost_function, gradient_function])\n\n cost = 0.0\n\n # In a loop, keep adjusting the hacked image slightly so that it tricks the model more and more\n # until it gets to at least 80% confidence\n while cost < 0.99:\n # Check how close the image is to our target class and grab the gradients we\n # can use to push it one more step in that direction.\n # Note: It's really important to pass in '0' for the Keras learning mode here!\n # Keras layers behave differently in prediction vs. train modes!\n cost, gradients = grab_cost_and_gradients_from_model([hacked_image, 0])\n\n # Move the hacked image one step further towards fooling the model\n # print gradients\n hacked_image += np.sign(gradients) * learning_rate\n\n # Ensure that the image doesn't ever change too much to either look funny or to become an invalid image\n hacked_image = np.clip(hacked_image, max_change_below, max_change_above)\n hacked_image = np.clip(hacked_image, -1.0, 1.0)\n\n print(\"Model's predicted likelihood that the image is a tree frog: {:.8}%\".format(cost * 100))\n\n hacked_image = hacked_image.reshape((224,224,3))\n img = array_to_img(hacked_image)\n img.save(img_res_path)","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:42:05.713062Z","iopub.execute_input":"2022-09-16T21:42:05.713693Z","iopub.status.idle":"2022-09-16T21:42:05.731040Z","shell.execute_reply.started":"2022-09-16T21:42:05.713647Z","shell.execute_reply":"2022-09-16T21:42:05.729556Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"IMAGE_DIMS = (224, 224)\nTREE_FROG_IDX = 31\nTREE_FROG_STR = \"tree_frog\"\n\nimg_path = data\nimg_res_path = './'\nmodel_path = Im.open(f\"{dir_name}/salt/turtle.jpg\")\n# \ntarget_str = 31\ntarget_idx = \"tree_frog\"\n\n\ncreate_img(img_path, img_res_path, model_path, target_str, target_idx, des_conf=0.95)","metadata":{"execution":{"iopub.status.busy":"2022-09-13T20:18:10.929394Z","iopub.execute_input":"2022-09-13T20:18:10.930595Z","iopub.status.idle":"2022-09-13T20:18:11.785713Z","shell.execute_reply.started":"2022-09-13T20:18:10.930540Z","shell.execute_reply":"2022-09-13T20:18:11.783589Z"}}},{"cell_type":"code","source":"import pickle\nfile = open('/kaggle/input/ai-village-ctf/salt/picklemodel', 'rb')\n\n# dump information to that file\ndata = pickle.load(file)","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:42:05.732831Z","iopub.execute_input":"2022-09-16T21:42:05.733288Z","iopub.status.idle":"2022-09-16T21:42:06.257912Z","shell.execute_reply.started":"2022-09-16T21:42:05.733246Z","shell.execute_reply":"2022-09-16T21:42:06.256684Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from keras.applications.mobilenet import preprocess_input\nfrom keras.models import load_model\nfrom keras.preprocessing.image import img_to_array, array_to_img\nfrom PIL import Image\nfrom imagehash import phash\nimport numpy as np\nfrom keras import backend as K","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:42:06.259776Z","iopub.execute_input":"2022-09-16T21:42:06.260612Z","iopub.status.idle":"2022-09-16T21:42:06.270612Z","shell.execute_reply.started":"2022-09-16T21:42:06.260563Z","shell.execute_reply":"2022-09-16T21:42:06.269462Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def submitforturtle(data):\n try:\n r = requests.post(\"https://salt.fly.dev/score\", data=json.dumps(data))\n return(r.text)\n except requests.exceptions.ConnectionError:\n return(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:42:06.272818Z","iopub.execute_input":"2022-09-16T21:42:06.273264Z","iopub.status.idle":"2022-09-16T21:42:06.281442Z","shell.execute_reply.started":"2022-09-16T21:42:06.273221Z","shell.execute_reply":"2022-09-16T21:42:06.280024Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:42:06.283821Z","iopub.execute_input":"2022-09-16T21:42:06.284280Z","iopub.status.idle":"2022-09-16T21:42:06.296071Z","shell.execute_reply.started":"2022-09-16T21:42:06.284238Z","shell.execute_reply":"2022-09-16T21:42:06.294718Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\nIMAGE_DIMS = (224, 224)\n\nimg = Im.open(f\"{dir_name}/salt/turtle.jpg\")\nimg = img.resize(IMAGE_DIMS)\nimg_bytes = img.tobytes()\nb64_img_bytes = base64.urlsafe_b64encode(img_bytes)\n\ndata = {\n \"input\": b64_img_bytes.decode()\n}\n\nprint(submitforturtle(data))","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:42:06.302045Z","iopub.execute_input":"2022-09-16T21:42:06.302782Z","iopub.status.idle":"2022-09-16T21:42:06.906888Z","shell.execute_reply.started":"2022-09-16T21:42:06.302737Z","shell.execute_reply":"2022-09-16T21:42:06.905662Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Submission Helper\n\nIMAGE_DIMS = (224, 224)\n\n#img = Im.open(f\"owl.png\")\nimg = Im.open(f\"{dir_name}/salt/turtle.jpg\")\nimg = img.resize(IMAGE_DIMS)\nimg_bytes = img.tobytes()\nb64_img_bytes = base64.urlsafe_b64encode(img_bytes)\n\ndata = {\n \"input\": b64_img_bytes.decode()\n}\n\ntry:\n r = requests.post(\"https://salt.fly.dev/score\", data=json.dumps(data))\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")\n \nmy_flag = json.loads(r.text)['message'][-128:]","metadata":{"execution":{"iopub.status.busy":"2022-09-15T13:08:08.365986Z","iopub.execute_input":"2022-09-15T13:08:08.366421Z","iopub.status.idle":"2022-09-15T13:08:09.036141Z","shell.execute_reply.started":"2022-09-15T13:08:08.366388Z","shell.execute_reply":"2022-09-15T13:08:09.034524Z"}}},{"cell_type":"code","source":"add_flag(\"salt\", \"3XK5MLIC73A4TML4CVCTKRLAN2\n\n# Crop1 (id: crop1) -- 400 Points\n\nI never understood how this even worked. Poisoned Red circle? I don't see that. I needed to spend more time walking back through our code.\n\n\n\n\n\n\nFavorite solution:\n [.894](https://www.kaggle.com/code/johnmacgillivray/894-solutions#Crop1-(id:-crop1)----400-Points)","metadata":{}},{"cell_type":"code","source":"!ls /kaggle/input/ai-village-ctf/crop","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:42:06.927757Z","iopub.execute_input":"2022-09-16T21:42:06.928220Z","iopub.status.idle":"2022-09-16T21:42:08.189869Z","shell.execute_reply.started":"2022-09-16T21:42:06.928177Z","shell.execute_reply":"2022-09-16T21:42:08.188199Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from IPython.display import Image\nImage(filename='/kaggle/input/ai-village-ctf/crop/sample_image.png')","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:42:08.192499Z","iopub.execute_input":"2022-09-16T21:42:08.193001Z","iopub.status.idle":"2022-09-16T21:42:08.211015Z","shell.execute_reply.started":"2022-09-16T21:42:08.192952Z","shell.execute_reply":"2022-09-16T21:42:08.209803Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!cat /kaggle/input/ai-village-ctf/crop/crop2_example.json","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:42:08.213017Z","iopub.execute_input":"2022-09-16T21:42:08.213491Z","iopub.status.idle":"2022-09-16T21:42:09.274785Z","shell.execute_reply.started":"2022-09-16T21:42:08.213447Z","shell.execute_reply":"2022-09-16T21:42:09.273404Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = joblib.load(f\"{dir_name}/crop/04JUN_crop_model.pckl\")\nX_comp = joblib.load(f\"{dir_name}/crop/X_comp.pckl\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:42:09.276259Z","iopub.execute_input":"2022-09-16T21:42:09.276638Z","iopub.status.idle":"2022-09-16T21:42:24.718822Z","shell.execute_reply.started":"2022-09-16T21:42:09.276601Z","shell.execute_reply":"2022-09-16T21:42:24.717407Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Helper Functions\ndef get_ij(idx):\n idx = 8-idx\n j = idx//3\n i = 2-idx%3\n return i, j\n\ndef resize(X, size=(224,224)):\n X_new = np.array(Im.fromarray(X.astype(np.uint8)).resize(size, resample=3))\n return X_new\n\ndef get_cropped(X, idx):\n n = 75*3\n i, j = get_ij(idx)\n crop = resize(X,(n,n))[j*75:(j+1)*75,i*75:(i+1)*75,:]\n return crop\n\ndef i2t(img:np.array):\n return np.array(img, dtype=np.float32)/255.\n\ndef make_X(x, size=(224,224)):\n side = int((len(x)//3)**0.5)\n x = np.where(x<255,x,255)\n x = np.where(x>0,x,0)\n X_np = x.astype(int).reshape([side, side, 3])\n X_np = np.array(Im.fromarray(X_np.astype(np.uint8)).resize(size, resample=Im.Resampling.NEAREST))\n return X_np\n\ndef pt_ft(X_np):\n return torch.FloatTensor(X_np)\n\ndef score(x, model=model, X_comp = np.divide(X_comp,0b1010), debug=False):\n x = (x).astype(int)\n model.eval()\n X = make_X(x)\n preds = model(pt_ft(i2t(X.T[None])))\n idx = preds.argmax().item()\n expected = ((25.5-X_comp)).flatten()\n actual_square = get_cropped(X, idx)\n actual = actual_square.flatten().astype(int)\n if debug:\n print({\"idx\":idx})\n plt.imshow(actual_square)\n plt.show()\n sse = ((actual-(expected*10).astype(int))**2).sum()\n redness = (actual_square[:,:,0][actual_square[:,:,0]>230]**2).sum()+1\n if idx==8:\n return (1+preds.max().item())*sse*redness\n return sse*redness\n\ndef get_submittable(x):\n return \",\".join([str(int(n)) for n in x])\n\ncfn = lambda x: [f\"background-color: RGB({x.R},{x.G},{x.B})\"] * 3\ndef show_colors(ans):\n ans = [int(x) for x in ans.split(\",\")]\n squares = pd.DataFrame(np.array(ans[:27]).reshape([9,3]), columns=[\"R\",\"G\",\"B\"]).T.style.apply(cfn, axis=0)\n circles = pd.DataFrame(np.array(ans[27:]).reshape([9,3]), columns=[\"R\",\"G\",\"B\"]).T.style.apply(cfn, axis=0)\n display(squares)\n display(circles)","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:42:24.724153Z","iopub.execute_input":"2022-09-16T21:42:24.724885Z","iopub.status.idle":"2022-09-16T21:42:24.748212Z","shell.execute_reply.started":"2022-09-16T21:42:24.724831Z","shell.execute_reply":"2022-09-16T21:42:24.747257Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_small = np.random.normal(100,30,size=(5**2*3)).reshape((5,5,3))","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:42:24.749990Z","iopub.execute_input":"2022-09-16T21:42:24.750849Z","iopub.status.idle":"2022-09-16T21:42:24.765404Z","shell.execute_reply.started":"2022-09-16T21:42:24.750806Z","shell.execute_reply":"2022-09-16T21:42:24.763914Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"inter=np.divide(X_comp,0b1010)\n\nplt.imshow(inter)","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:42:24.767259Z","iopub.execute_input":"2022-09-16T21:42:24.768116Z","iopub.status.idle":"2022-09-16T21:42:25.033364Z","shell.execute_reply.started":"2022-09-16T21:42:24.768069Z","shell.execute_reply":"2022-09-16T21:42:25.032039Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from PIL import Image, ImageDraw\nfrom random import randint, randrange\n\n\ndef circle(draw: ImageDraw, size: int) -> None:\n #xc, yc = randrange(size), randrange(size)\n #radius = randrange(size) / randint(5, 10)\n xc,yc=115,112\n radius=25\n color = 179,222,105#randrange(256), randrange(256), randrange(256)\n\n draw.ellipse(\n xy=(\n xc - radius, yc - radius,\n xc + radius, yc + radius,\n ),\n fill=color,\n )\n \n xc,yc=27,25\n radius=25\n color = 179,222,105#randrange(256), randrange(256), randrange(256)\n\n draw.ellipse(\n xy=(\n xc - radius, yc - radius,\n xc + radius, yc + radius,\n ),\n fill=color,\n )\n \n xc,yc=109,187\n radius=25\n color = 179,222,105#randrange(256), randrange(256), randrange(256)\n\n draw.ellipse(\n xy=(\n xc - radius, yc - radius,\n xc + radius, yc + radius,\n ),\n fill=color,\n )\n \n xc,yc=187,107\n radius=25\n color = 179,222,105#randrange(256), randrange(256), randrange(256)\n\n draw.ellipse(\n xy=(\n xc - radius, yc - radius,\n xc + radius, yc + radius,\n ),\n fill=color,\n )\n\n\n\ndef ttt():\n size = 224\n circle_amount = 1\n img: Image = Image.new(\"RGB\", (size, size), \"black\")\n for x in range(224):\n for y in range(224):\n #current_color = picture.getpixel( (x,y) )\n img.putpixel( (x,y), (220,128,114))\n draw: ImageDraw = ImageDraw.Draw(img)\n\n for _ in range(circle_amount):\n circle(draw, size)\n display(img)\n img.save(\"image.png\")\n return img\n\nimg=ttt()","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:42:25.035091Z","iopub.execute_input":"2022-09-16T21:42:25.036151Z","iopub.status.idle":"2022-09-16T21:42:25.133519Z","shell.execute_reply.started":"2022-09-16T21:42:25.036104Z","shell.execute_reply":"2022-09-16T21:42:25.132386Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from PIL import Image\nfrom numpy import asarray\n \n \n# load the image and convert into\n# numpy array\nimg = Image.open('image.png')\n# asarray() class is used to convert\n# PIL images into NumPy arrays\nn = asarray(img)\n\nim = np.array(get_submittable(n.flatten()).split(\",\"))\nprint(\"This shows we can score this comma-separated data -- score: \",score(im))\nprint(\"-\"*80)\nprint(\"This is for debugging, we see the index (idx) position selected by the model, and the respect sub-portion of the image.\")\nscore(n.flatten(), debug=True)","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:42:25.135198Z","iopub.execute_input":"2022-09-16T21:42:25.135921Z","iopub.status.idle":"2022-09-16T21:42:25.819416Z","shell.execute_reply.started":"2022-09-16T21:42:25.135873Z","shell.execute_reply":"2022-09-16T21:42:25.818368Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"try:\n r = requests.post(\"https://crop1.fly.dev/score\", data=json.dumps({\"data\": get_submittable(n.flatten())}))\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:42:25.824229Z","iopub.execute_input":"2022-09-16T21:42:25.825215Z","iopub.status.idle":"2022-09-16T21:42:26.354905Z","shell.execute_reply.started":"2022-09-16T21:42:25.825168Z","shell.execute_reply":"2022-09-16T21:42:26.353511Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"best=102505125\nbestseed=0\n\n\nfor i in range(1,5000,1):\n for i2 in range(4,10):\n for i3 in range(1,130,10):\n np.random.seed(i)\n X_small = np.random.normal(100,i3,size=(i2**2*3)).reshape((i2,i2,3))\n if (score(X_small.flatten())\n\n# Crop2 (id: crop2) -- 800 Points\nNever attempted.\n\nOnly solution:\n [(One) crop2 solution](https://www.kaggle.com/code/gladwell/one-crop2-solution)\n","metadata":{}},{"cell_type":"code","source":"import sys\nimport time\nfrom itertools import permutations, chain\nfrom collections import Counter\nimport warnings\nwarnings.filterwarnings('ignore')\n\n# third party imports\nimport joblib\nimport requests\nimport numpy as np\nimport json\nfrom PIL import Image, ImageDraw\nfrom torch import FloatTensor","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:42:26.376090Z","iopub.execute_input":"2022-09-16T21:42:26.376487Z","iopub.status.idle":"2022-09-16T21:42:26.387023Z","shell.execute_reply.started":"2022-09-16T21:42:26.376451Z","shell.execute_reply":"2022-09-16T21:42:26.385622Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class Generator(object):\n def __init__(self, hex_colours: list):\n self.hex_colours = hex_colours\n self.rgb_colours = [tuple(int(clr[i:(i + 2)], 16) for i in (1, 3, 5)) for clr in self.hex_colours]\n self.n_colours = len(self.rgb_colours)\n self.grid_size = int(self.n_colours**.5)\n self._generate_orders()\n\n def _chunkify(x, size):\n n_chunks = len(x)//size\n if len(x) % size != 0:\n n_chunks += 1\n return [x[(i * size):((i + 1) * size)] for i in range(n_chunks)]\n\n def _generate_orders(self):\n self.colour_orders = np.array(list(permutations(range(self.n_colours), self.n_colours)))\n self.squares = self.colour_orders.copy()\n self.circles = self.colour_orders.copy()\n return None\n \n def fix_square(self, square_index, colour_indexes):\n self.squares = self.squares[np.where(np.isin(self.squares[:,square_index], colour_indexes))]\n return None\n \n def fix_circle(self, circle_index, colour_indexes):\n self.circles = self.circles[np.where(np.isin(self.circles[:,circle_index], colour_indexes))]\n return None\n\n def check_size(self):\n print(f'{self.squares.shape[0]} x {self.circles.shape[0]} permutations remaining')\n return None\n\n def sample(self, n_samples=1):\n samples = []\n while len(samples) < n_samples:\n squares = self.squares[np.random.choice(self.squares.shape[0])].tolist()\n circles = self.circles[np.random.choice(self.circles.shape[0])].tolist()\n if any([s==c for s, c in zip(squares, circles)]):\n continue\n else:\n samples.append({'squares': squares, 'circles': circles})\n return samples\n \n def _square_coords(self, width):\n squares_xy = [\n [\n [\n (row * width, col * width),\n ((row + 1) * width, (col + 1) * width)\n ] for col in range(self.grid_size)\n ] for row in range(self.grid_size)\n ]\n return squares_xy\n \n def _circle_coords(self, width, radius):\n border = (width - (radius * 2)) / 2\n circles_xy = [\n [\n [\n (row * width + border, col * width + border),\n ((row + 1) * width - border, (col + 1) * width - border)\n ] for col in range(self.grid_size)\n ] for row in range(self.grid_size)\n ]\n return circles_xy\n \n def draw(self, sample: dict, width: int = 75, radius: int = 25, show: bool = False, output_size: tuple = (224, 224)):\n squares_xy = self._square_coords(width)\n circles_xy = self._circle_coords(width, radius)\n square_clrs = [self.rgb_colours[clr_index] for clr_index in sample['squares']]\n circle_clrs = [self.rgb_colours[clr_index] for clr_index in sample['circles']]\n img = Image.new(mode='RGB', size=(width * self.grid_size, width * self.grid_size), color=(255, 255, 255))\n draw = ImageDraw.Draw(img)\n for row in range(self.grid_size):\n for col in range(self.grid_size):\n draw.rectangle(xy = squares_xy[row][col], fill=tuple(square_clrs[row * self.grid_size + col]))\n draw.ellipse(xy = circles_xy[row][col], fill=tuple(circle_clrs[row * self.grid_size + col]))\n img = img.transpose(method=Image.TRANSVERSE).transpose(Image.FLIP_LEFT_RIGHT)\n img_resized = img.resize(output_size, resample=Image.NEAREST)\n if show:\n img_resized.show()\n img_out = np.array(img_resized).astype(int)\n return img_out\n \n def submit(self, sample):\n submission = {\n 'squares': list(chain(*[artist.rgb_colours[i] for i in sample[:9]])),\n 'circles': list(chain(*[artist.rgb_colours[i] for i in sample[9:]]))\n }\n try:\n response = requests.post(\"https://crop2.fly.dev/score\", json=submission)\n if response.json()['message'] != [\"That doesn't seem quite right. Try again.\", 400]:\n print(f'\\n{response.text}\\n{sample}\\n')\n except requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")\n response = None\n return response","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:42:26.389121Z","iopub.execute_input":"2022-09-16T21:42:26.389974Z","iopub.status.idle":"2022-09-16T21:42:26.419328Z","shell.execute_reply.started":"2022-09-16T21:42:26.389925Z","shell.execute_reply":"2022-09-16T21:42:26.417653Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class Scorer(object):\n def __init__(self, model_path):\n self.model_path = model_path\n self._load_model()\n self.sample_dict = {}\n \n def _load_model(self):\n self.model = joblib.load(self.model_path)\n _ = self.model.eval()\n \n def i2t(self, img:np.array):\n return np.array(img, dtype=np.float32)/255.\n\n def make_X(self, x, size=(224,224)):\n side = int((len(x)//3)**0.5)\n x = np.where(x<255,x,255)\n x = np.where(x>0,x,0)\n X_np = x.astype(int).reshape([side, side, 3])\n X_np = np.array(Image.fromarray(X_np.astype(np.uint8)).resize(size, resample=Image.NEAREST))\n return X_np\n\n def pt_ft(self, X_np):\n return FloatTensor(X_np)\n \n def evaluate(self, reference, image): \n reference = tuple(chain(*reference.values()))\n if reference in self.sample_dict.keys():\n predictions = self.sample_dict[reference]\n else:\n flat_image = image.flatten()\n input_tensor = self.pt_ft(self.i2t(self.make_X(flat_image).T[None]))\n predictions = tuple(self.model(input_tensor).detach().numpy()[0])\n self.sample_dict[reference] = predictions\n return predictions\n \n def column_rates(self, array, n_top=8):\n n_rows = array.shape[0]\n n_cols = array.shape[1]\n prev_id = {}\n for col_num in range(n_cols):\n prev_id[col_num] = []\n counter_items = Counter(array[:,col_num]).most_common(n_top)\n for value, count in counter_items:\n prev_id[col_num].append((round(count/n_rows, 3), value))\n return prev_id\n\n def best_samples_by_index(self, target_idx=8, n_top=10):\n threshold = sorted(np.array(list(self.sample_dict.values()))[:,target_idx], reverse=True)[n_top]\n best_samples = np.array([k for k, v in self.sample_dict.items() if v[target_idx] >= threshold])\n return best_samples, threshold","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:42:26.421410Z","iopub.execute_input":"2022-09-16T21:42:26.422359Z","iopub.status.idle":"2022-09-16T21:42:26.440918Z","shell.execute_reply.started":"2022-09-16T21:42:26.422286Z","shell.execute_reply":"2022-09-16T21:42:26.439972Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"np.random.seed(42)\nartist = Generator(['#8DD3C7', '#FFFFB3', '#BEBADA', '#FB8072', '#80B1D3', '#FDB462', '#B3DE69', '#FCCDE5', '#D9D9D9'])\ncritic = Scorer('/kaggle/input/ai-village-ctf/crop/04JUN_crop_model.pckl')","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:42:26.442513Z","iopub.execute_input":"2022-09-16T21:42:26.443232Z","iopub.status.idle":"2022-09-16T21:42:27.780029Z","shell.execute_reply.started":"2022-09-16T21:42:26.443177Z","shell.execute_reply":"2022-09-16T21:42:27.778705Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# The poisoned sample is almost certainly idx8\n# Fix the colours for idx8 to the colours of the ideal answer from crop1\n# since that cropped to a non-red square\n# Any permutation without these colours in idx8 are removed with these actions\nartist.fix_square(8, [3])\nartist.fix_circle(8, [6])\n\n# We're only interested in samples that score very highly on idx8 (top 100)\n# We know the model was exposed to this sample many times\n# So it should be very high scoring on idx8\nn_top = 100\n\n# Any colour that it less than 5% likely to be in a position is removed from the seaarch space\n# We need to be forgiving here with the fairly low threshold\n# as there may be other legitimate idx8 examples around the same feature space\nprevalence_threshold = 0.05","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:42:27.781781Z","iopub.execute_input":"2022-09-16T21:42:27.782175Z","iopub.status.idle":"2022-09-16T21:42:27.799180Z","shell.execute_reply.started":"2022-09-16T21:42:27.782141Z","shell.execute_reply":"2022-09-16T21:42:27.797744Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# In the loop below, we sample randomly from the set of fixed permutations\n# Each sample gets evaluated and every 500 we look for more colour(s) to exclude\n# Then we submit all of the top 100 of what we have to the scoring endpoint\n# Each time we manage to rule out more colours, the search space reduces massively\n#n_iterations = 10000\nn_iterations = 0\n\nflag_found = False\nstart_time = time.time()\nfor iter in range(n_iterations):\n sample = artist.sample()[0]\n image = artist.draw(sample)\n predictions = critic.evaluate(sample, image)\n print(f'\\r{iter}/{n_iterations}: {(time.time()-start_time):.1f}s elapsed', end='')\n if iter % 500 == 0 and iter > 0:\n best_samples, threshold = critic.best_samples_by_index(n_top=n_top)\n print(f' - Submitting {len(best_samples)} samples at threshold {threshold:.4f}')\n for submittable_sample in best_samples:\n response = artist.submit(submittable_sample)\n if 'You are the cropping champion.' in response.text:\n with open('flag.txt', 'w') as f:\n f.writelines(response.text)\n flag_found = True\n column_stats = critic.column_rates(best_samples)\n for index, rate_clr in column_stats.items():\n clrs = [clr for rate, clr in rate_clr if rate > prevalence_threshold]\n if index > 8:\n artist.fix_circle(index - 9, clrs)\n else:\n artist.fix_square(index, clrs)\n artist.check_size()\n if flag_found:\n break","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:42:27.800898Z","iopub.execute_input":"2022-09-16T21:42:27.801326Z","iopub.status.idle":"2022-09-16T21:42:27.812651Z","shell.execute_reply.started":"2022-09-16T21:42:27.801280Z","shell.execute_reply":"2022-09-16T21:42:27.811110Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Submission Helper\n\nwith open(f\"{dir_name}/crop/crop2_example.json\", \"r\") as f:\n test = json.load(f)\n\ntry:\n r = requests.post(\"https://crop2.fly.dev/score\", json=test)\n print(r.text)\nexcept requests.exceptions.ConnectionError:\n print(\"Connection problems. Contact the CTF organizers.\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:42:27.814944Z","iopub.execute_input":"2022-09-16T21:42:27.815456Z","iopub.status.idle":"2022-09-16T21:42:28.195237Z","shell.execute_reply.started":"2022-09-16T21:42:27.815406Z","shell.execute_reply":"2022-09-16T21:42:28.193926Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"add_flag(\"crop2\", \"NUouV7DXFy80lZlFW6lVSY3PLVQJhVIMRO11476lYPWeF4HLu4FV3kYlZXSDiMR8HPJoN7JDTTW2MCJHWQBeACQBKAJT{3M9FDd41Der0XeUn06MOJUWZFGs}DFTGBN1\")","metadata":{"execution":{"iopub.status.busy":"2022-09-16T21:42:28.197469Z","iopub.execute_input":"2022-09-16T21:42:28.198216Z","iopub.status.idle":"2022-09-16T21:42:28.209896Z","shell.execute_reply.started":"2022-09-16T21:42:28.198168Z","shell.execute_reply":"2022-09-16T21:42:28.208633Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}