{
  "id": 351840,
  "title": "10th place write up",
  "url": "/competitions/ai-village-ctf/writeups/john-macgillivray-10th-place-write-up",
  "author_name": "",
  "post_date": "2022-09-12T04:44:53.400Z",
  "votes": 9,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Thankyou to Kaggle, AI Village, and my competitors for an unbelievably fun competition. I was very lucky that I was not working for 2 weeks when I found this competition, so I could become utterly obsessed and throw every waking hour at it. </p>\n<p>Solutions notebook published <a href=\"https://www.kaggle.com/code/johnmacgillivray/894-solutions\" target=\"_blank\">here</a></p>\n<p>Competition feedback provided <a href=\"https://www.kaggle.com/competitions/ai-village-ctf/discussion/346451#1935337\" target=\"_blank\">here</a> </p>\n<p><strong>Thoughts on each problem:</strong></p>\n<p><strong>Hotdog:</strong> Got a picture of an actual hotdog from a dataset on kaggle for hotdog not hotdog, the ultimate classification use case.</p>\n<p><strong>Math 1-4:</strong> Was already vaguely familiar with these topics so googled around to remind myself how to do it and basically followed the instructions.</p>\n<p><strong>WIFI:</strong> I won't lie this one took me a depressing amount of time to get, but in the end I tried sorting the tokens by the max argument of their associated embeddings and that did the trick.</p>\n<p><strong>HOTTERDOG:</strong> I used the same solution as for THEFT and SALT, see them for details.</p>\n<p><strong>Honorstudent:</strong> I crudely blacked out some pixels and white pixels until the A I drew passed, very by hand, but rewarding nonetheless!</p>\n<p><strong>Secret Sloth:</strong> I bashed my head against the wall for many days trying a million things, and then the competition hosts hinted at signal processing, and I tried fourier transform, but because I was trying so many things in such quick succession my attempt was sloppy in that i didnt do all the steps, and even then, i wouldnt have played with it enough to see the part of the image you needed to see. After circling back to it and trying again however, 5 lines of default inverse fourier transform being careful with the scaling got me the flag. </p>\n<p>I also found the original image on reddit and then started doing all the things that failed to work on both original image and encrypted image and looking at the deltas. This lead to a troubling set of images where I was convinced I could almost see letters. This was quite an unhinged period of the competition for me.</p>\n<p><img src=\"https://cdn.discordapp.com/attachments/1011651849880875159/1011653051582525460/unknown.png\" alt=\"my suffering\"></p>\n<p><strong>Bad to Good:</strong>  This one was really fun, I tried a few intuitive ideas about making Henry better and classmates worse, but to no avail. I then tried letting it do random walks just to see what would get a better score, and my naive coding led to the best walks were when Henry had a negative number of demerits. This gave me the dumb idea of 'well if its most sensitive to this and doesn't take negative amounts as illegal, lets just give Henry -20 demerits and nothing else' and it worked. Fun challenge. </p>\n<p><strong>Inference:</strong>  This one was hard. I got three different datasets of written and typed characters, and started throwing them at the models to see which letters scored most highly for each of the 6 outputs. At first I was just looking for any letters which hit over a threshold, with un-flipped images. This was a bad approach in that to get any useful letter matching you have to set your threshold very high, and without flipping them you get red herrings and miss the good letters. As you will see from my solution code, what worked best for me was a climber, where to spit out a candidate image as a letter you have to be the best one yet. Another key to this was flipping the images, as was implied by the hints in the question. After some deranged playing with the candidate letters in excel, I worked out what it had to be. Took a lot of work felt good to get it right.</p>\n<p><strong>Baseball:</strong>  A fun one that didn't take too long, just played around with trial and error almost by feel, working out what parameters to tweak in real time like I was playing hot/cold.</p>\n<p><strong>WAF:</strong> This was great, a very educational research style puzzle. I got from the hint example string provided by hosts that this was base 64 converted and that the valid string they were getting at was 'bash'. I did not know much bash, so this lead to many hours of googling and trying stuff. I understood my goal was to write a bash command that would convince the WAF that the string I had entered was a benign string, but was struggling to implement it in a way that it actually gave me a flag rather than either triggering the WAF detection or failing to do anything cool and being a benign string. Eventually my googling led me to a specific well documented weakness that bash could exploit: 'shellshock', and code which tested exposure to that got it to work. Links in my solution code. </p>\n<p><strong>LEAKAGE:</strong> I will not lie, this one exposed my lack of familiarity with LSTM and this type of model. I had to google for a few hours, and even then, I at first was stupidly just trying to go straight for the LSTM layer with my featurized vector instead of pushing it through all the layers in the model, which meant I was getting gibberish out XD. But after a day or two of staring at the boilerplate code provided I worked out what you had to do. So this was humbling and reminded me how little I know about data science, but also rewarding when I got it to work.</p>\n<p><strong>FORENSICS:</strong> This was a nice quick one for me, just look at the model metadata / parameters however you want to describe it. Still educational though as it shows you how you can learn things about a model if you were trying to hack it and look for vulnerabilities, or even just trying to understand how to use it and plug it into something and make it work.</p>\n<p><strong>THEFT:</strong> So this one was deceptively hard for 100 points, however as I said earlier, cracking this one instantly got me hotterdog and salt: I failed to do anything with the file provided, however by peaking at the parameters of the salt model I worked out it was a mobilnet v2. So I grabbed the pretrained mobilnet v2 from online and used the code linked to by the competition hosts on how to do this, tweaker the code as needed and changed the goal category, and after 10 mins of iterating it worked! And this exact method was used to also hotterdog and salt in quick succession (for salt I used the model provided FWIW, but same principle.)</p>\n<p><strong>SALT:</strong> See theft.</p>\n<p><strong>TOKEN:</strong> So this was clearly a well thought out and 'hard-if-you-dont-look-for-the-specific-thing' problem. I tried looking for all sorts of specific character combos by hand in excel and python but nothing I tried worked, and after a few hours of hitting dead ends, with 4 problems to go, and having not even read or attempted WAF yet, I decided to set off a barrage of notebooks brute-forcing token whilst I worked on WAF. Sure enough after many hours of hammering the API I got the answer, and saw that the killer thing to look for was 'BLANK BLANK'. Was kicking myself I didn't try to look for that. Anyway, I don't think I'd have got it without it being brute forcible (or it would have taken my WAY longer that's for sure). </p>\n<p><strong>CROP1</strong>: This was nice, got it after a bit of trying, I played very specifically with the reference image and goal-seeked by hand the pixel colours etc., and simply added circles in the right places until the poisoned cropping model cropped to one of the circles I wanted it to crop to. No random walks or anything just understanding every line of the scoring function.</p>\n<p><strong>CROP2:</strong> Tried a bunch of stuff, none of it worked, looking forward to seeing a working solution!</p>\n<p><strong>DeepFake:</strong> The first dumb thing I tried of grabbing a video of same length worked. Which I won't lie felt good, however also realized that was not the intent of the author, and I bet it would have been super interesting and fun and rewarding to learn how to actually reverse deepfakes (or maybe next competition, to implement a deepfake? :) ) </p>\n<p><strong>MURDERBOTS:</strong> This is a classic bread and butter ML problem: 'here is a test set, here is a training set, build an accurate model and prove to us it's good by submitting your predictions for the test set'. Got it using code I've used tons of times in other similar questions, but was a great fun question which will have been educational for people not used to ML or this specific concept.</p>\n<p><strong>Rating problem difficulties</strong></p>\n<p>In terms of total difficulties, I decided the fairest metric would be how long it took me to break each problem (note - all of the problems were great and interesting and fun to crack, and this is just how long it personally took me - some questions I got hung up on just because of my own silly mistakes, others I got lucky cracking quickly):</p>\n<p>Easiest: Math 1-4, Honorstudent, Baseball, Forensics, Hotdog, Crop1, Deepfake, Murderbots<br>\nMedium: WIFI, Theft/Salt/Hotterdog, Bad to good, WAF, Leakage<br>\nHard: Inference, Token, Secret Sloth<br>\nFailed to do: Crop2</p>\n<p><strong>Final thoughts</strong></p>\n<p>So all in all - great fun, loved every second of the competition. Also credit to IsaiahP who got 21/22 flags in scarily quick time way ahead of anyone else without hints, deserving winner imho.</p>\n<p>Will see you next year hopefully but unfortunately I will likely not have as much time to sink in so I probably won't do as well from a rank perspective, but I bet I'll still have lots of fun playing around and cracking a few good puzzles.</p>",
  "messages": [
    {
      "id": "1935391",
      "postDate": "09/12/2022 04:41:45",
      "content": "<p>Thankyou to Kaggle, AI Village, and my competitors for an unbelievably fun competition. I was very lucky that I was not working for 2 weeks when I found this competition, so I could become utterly obsessed and throw every waking hour at it. </p>\n<p>Solutions notebook published <a href=\"https://www.kaggle.com/code/johnmacgillivray/894-solutions\" target=\"_blank\">here</a></p>\n<p>Competition feedback provided <a href=\"https://www.kaggle.com/competitions/ai-village-ctf/discussion/346451#1935337\" target=\"_blank\">here</a> </p>\n<p><strong>Thoughts on each problem:</strong></p>\n<p><strong>Hotdog:</strong> Got a picture of an actual hotdog from a dataset on kaggle for hotdog not hotdog, the ultimate classification use case.</p>\n<p><strong>Math 1-4:</strong> Was already vaguely familiar with these topics so googled around to remind myself how to do it and basically followed the instructions.</p>\n<p><strong>WIFI:</strong> I won't lie this one took me a depressing amount of time to get, but in the end I tried sorting the tokens by the max argument of their associated embeddings and that did the trick.</p>\n<p><strong>HOTTERDOG:</strong> I used the same solution as for THEFT and SALT, see them for details.</p>\n<p><strong>Honorstudent:</strong> I crudely blacked out some pixels and white pixels until the A I drew passed, very by hand, but rewarding nonetheless!</p>\n<p><strong>Secret Sloth:</strong> I bashed my head against the wall for many days trying a million things, and then the competition hosts hinted at signal processing, and I tried fourier transform, but because I was trying so many things in such quick succession my attempt was sloppy in that i didnt do all the steps, and even then, i wouldnt have played with it enough to see the part of the image you needed to see. After circling back to it and trying again however, 5 lines of default inverse fourier transform being careful with the scaling got me the flag. </p>\n<p>I also found the original image on reddit and then started doing all the things that failed to work on both original image and encrypted image and looking at the deltas. This lead to a troubling set of images where I was convinced I could almost see letters. This was quite an unhinged period of the competition for me.</p>\n<p><img src=\"https://cdn.discordapp.com/attachments/1011651849880875159/1011653051582525460/unknown.png\" alt=\"my suffering\"></p>\n<p><strong>Bad to Good:</strong>  This one was really fun, I tried a few intuitive ideas about making Henry better and classmates worse, but to no avail. I then tried letting it do random walks just to see what would get a better score, and my naive coding led to the best walks were when Henry had a negative number of demerits. This gave me the dumb idea of 'well if its most sensitive to this and doesn't take negative amounts as illegal, lets just give Henry -20 demerits and nothing else' and it worked. Fun challenge. </p>\n<p><strong>Inference:</strong>  This one was hard. I got three different datasets of written and typed characters, and started throwing them at the models to see which letters scored most highly for each of the 6 outputs. At first I was just looking for any letters which hit over a threshold, with un-flipped images. This was a bad approach in that to get any useful letter matching you have to set your threshold very high, and without flipping them you get red herrings and miss the good letters. As you will see from my solution code, what worked best for me was a climber, where to spit out a candidate image as a letter you have to be the best one yet. Another key to this was flipping the images, as was implied by the hints in the question. After some deranged playing with the candidate letters in excel, I worked out what it had to be. Took a lot of work felt good to get it right.</p>\n<p><strong>Baseball:</strong>  A fun one that didn't take too long, just played around with trial and error almost by feel, working out what parameters to tweak in real time like I was playing hot/cold.</p>\n<p><strong>WAF:</strong> This was great, a very educational research style puzzle. I got from the hint example string provided by hosts that this was base 64 converted and that the valid string they were getting at was 'bash'. I did not know much bash, so this lead to many hours of googling and trying stuff. I understood my goal was to write a bash command that would convince the WAF that the string I had entered was a benign string, but was struggling to implement it in a way that it actually gave me a flag rather than either triggering the WAF detection or failing to do anything cool and being a benign string. Eventually my googling led me to a specific well documented weakness that bash could exploit: 'shellshock', and code which tested exposure to that got it to work. Links in my solution code. </p>\n<p><strong>LEAKAGE:</strong> I will not lie, this one exposed my lack of familiarity with LSTM and this type of model. I had to google for a few hours, and even then, I at first was stupidly just trying to go straight for the LSTM layer with my featurized vector instead of pushing it through all the layers in the model, which meant I was getting gibberish out XD. But after a day or two of staring at the boilerplate code provided I worked out what you had to do. So this was humbling and reminded me how little I know about data science, but also rewarding when I got it to work.</p>\n<p><strong>FORENSICS:</strong> This was a nice quick one for me, just look at the model metadata / parameters however you want to describe it. Still educational though as it shows you how you can learn things about a model if you were trying to hack it and look for vulnerabilities, or even just trying to understand how to use it and plug it into something and make it work.</p>\n<p><strong>THEFT:</strong> So this one was deceptively hard for 100 points, however as I said earlier, cracking this one instantly got me hotterdog and salt: I failed to do anything with the file provided, however by peaking at the parameters of the salt model I worked out it was a mobilnet v2. So I grabbed the pretrained mobilnet v2 from online and used the code linked to by the competition hosts on how to do this, tweaker the code as needed and changed the goal category, and after 10 mins of iterating it worked! And this exact method was used to also hotterdog and salt in quick succession (for salt I used the model provided FWIW, but same principle.)</p>\n<p><strong>SALT:</strong> See theft.</p>\n<p><strong>TOKEN:</strong> So this was clearly a well thought out and 'hard-if-you-dont-look-for-the-specific-thing' problem. I tried looking for all sorts of specific character combos by hand in excel and python but nothing I tried worked, and after a few hours of hitting dead ends, with 4 problems to go, and having not even read or attempted WAF yet, I decided to set off a barrage of notebooks brute-forcing token whilst I worked on WAF. Sure enough after many hours of hammering the API I got the answer, and saw that the killer thing to look for was 'BLANK BLANK'. Was kicking myself I didn't try to look for that. Anyway, I don't think I'd have got it without it being brute forcible (or it would have taken my WAY longer that's for sure). </p>\n<p><strong>CROP1</strong>: This was nice, got it after a bit of trying, I played very specifically with the reference image and goal-seeked by hand the pixel colours etc., and simply added circles in the right places until the poisoned cropping model cropped to one of the circles I wanted it to crop to. No random walks or anything just understanding every line of the scoring function.</p>\n<p><strong>CROP2:</strong> Tried a bunch of stuff, none of it worked, looking forward to seeing a working solution!</p>\n<p><strong>DeepFake:</strong> The first dumb thing I tried of grabbing a video of same length worked. Which I won't lie felt good, however also realized that was not the intent of the author, and I bet it would have been super interesting and fun and rewarding to learn how to actually reverse deepfakes (or maybe next competition, to implement a deepfake? :) ) </p>\n<p><strong>MURDERBOTS:</strong> This is a classic bread and butter ML problem: 'here is a test set, here is a training set, build an accurate model and prove to us it's good by submitting your predictions for the test set'. Got it using code I've used tons of times in other similar questions, but was a great fun question which will have been educational for people not used to ML or this specific concept.</p>\n<p><strong>Rating problem difficulties</strong></p>\n<p>In terms of total difficulties, I decided the fairest metric would be how long it took me to break each problem (note - all of the problems were great and interesting and fun to crack, and this is just how long it personally took me - some questions I got hung up on just because of my own silly mistakes, others I got lucky cracking quickly):</p>\n<p>Easiest: Math 1-4, Honorstudent, Baseball, Forensics, Hotdog, Crop1, Deepfake, Murderbots<br>\nMedium: WIFI, Theft/Salt/Hotterdog, Bad to good, WAF, Leakage<br>\nHard: Inference, Token, Secret Sloth<br>\nFailed to do: Crop2</p>\n<p><strong>Final thoughts</strong></p>\n<p>So all in all - great fun, loved every second of the competition. Also credit to IsaiahP who got 21/22 flags in scarily quick time way ahead of anyone else without hints, deserving winner imho.</p>\n<p>Will see you next year hopefully but unfortunately I will likely not have as much time to sink in so I probably won't do as well from a rank perspective, but I bet I'll still have lots of fun playing around and cracking a few good puzzles.</p>",
      "rawMarkdown": "Thankyou to Kaggle, AI Village, and my competitors for an unbelievably fun competition. I was very lucky that I was not working for 2 weeks when I found this competition, so I could become utterly obsessed and throw every waking hour at it. \n\nSolutions notebook published [here](https://www.kaggle.com/code/johnmacgillivray/894-solutions)\n\nCompetition feedback provided [here](https://www.kaggle.com/competitions/ai-village-ctf/discussion/346451#1935337) \n\n**Thoughts on each problem:**\n\n**Hotdog:** Got a picture of an actual hotdog from a dataset on kaggle for hotdog not hotdog, the ultimate classification use case.\n\n**Math 1-4:** Was already vaguely familiar with these topics so googled around to remind myself how to do it and basically followed the instructions.\n\n**WIFI:** I won't lie this one took me a depressing amount of time to get, but in the end I tried sorting the tokens by the max argument of their associated embeddings and that did the trick.\n\n**HOTTERDOG:** I used the same solution as for THEFT and SALT, see them for details.\n\n**Honorstudent:** I crudely blacked out some pixels and white pixels until the A I drew passed, very by hand, but rewarding nonetheless!\n\n**Secret Sloth:** I bashed my head against the wall for many days trying a million things, and then the competition hosts hinted at signal processing, and I tried fourier transform, but because I was trying so many things in such quick succession my attempt was sloppy in that i didnt do all the steps, and even then, i wouldnt have played with it enough to see the part of the image you needed to see. After circling back to it and trying again however, 5 lines of default inverse fourier transform being careful with the scaling got me the flag. \n\nI also found the original image on reddit and then started doing all the things that failed to work on both original image and encrypted image and looking at the deltas. This lead to a troubling set of images where I was convinced I could almost see letters. This was quite an unhinged period of the competition for me.\n\n![my suffering](https://cdn.discordapp.com/attachments/1011651849880875159/1011653051582525460/unknown.png)\n\n\n**Bad to Good:**  This one was really fun, I tried a few intuitive ideas about making Henry better and classmates worse, but to no avail. I then tried letting it do random walks just to see what would get a better score, and my naive coding led to the best walks were when Henry had a negative number of demerits. This gave me the dumb idea of 'well if its most sensitive to this and doesn't take negative amounts as illegal, lets just give Henry -20 demerits and nothing else' and it worked. Fun challenge. \n\n**Inference:**  This one was hard. I got three different datasets of written and typed characters, and started throwing them at the models to see which letters scored most highly for each of the 6 outputs. At first I was just looking for any letters which hit over a threshold, with un-flipped images. This was a bad approach in that to get any useful letter matching you have to set your threshold very high, and without flipping them you get red herrings and miss the good letters. As you will see from my solution code, what worked best for me was a climber, where to spit out a candidate image as a letter you have to be the best one yet. Another key to this was flipping the images, as was implied by the hints in the question. After some deranged playing with the candidate letters in excel, I worked out what it had to be. Took a lot of work felt good to get it right.\n\n**Baseball:**  A fun one that didn't take too long, just played around with trial and error almost by feel, working out what parameters to tweak in real time like I was playing hot/cold.\n\n**WAF:** This was great, a very educational research style puzzle. I got from the hint example string provided by hosts that this was base 64 converted and that the valid string they were getting at was 'bash'. I did not know much bash, so this lead to many hours of googling and trying stuff. I understood my goal was to write a bash command that would convince the WAF that the string I had entered was a benign string, but was struggling to implement it in a way that it actually gave me a flag rather than either triggering the WAF detection or failing to do anything cool and being a benign string. Eventually my googling led me to a specific well documented weakness that bash could exploit: 'shellshock', and code which tested exposure to that got it to work. Links in my solution code. \n\n**LEAKAGE:** I will not lie, this one exposed my lack of familiarity with LSTM and this type of model. I had to google for a few hours, and even then, I at first was stupidly just trying to go straight for the LSTM layer with my featurized vector instead of pushing it through all the layers in the model, which meant I was getting gibberish out XD. But after a day or two of staring at the boilerplate code provided I worked out what you had to do. So this was humbling and reminded me how little I know about data science, but also rewarding when I got it to work.\n\n**FORENSICS:** This was a nice quick one for me, just look at the model metadata / parameters however you want to describe it. Still educational though as it shows you how you can learn things about a model if you were trying to hack it and look for vulnerabilities, or even just trying to understand how to use it and plug it into something and make it work.\n\n**THEFT:** So this one was deceptively hard for 100 points, however as I said earlier, cracking this one instantly got me hotterdog and salt: I failed to do anything with the file provided, however by peaking at the parameters of the salt model I worked out it was a mobilnet v2. So I grabbed the pretrained mobilnet v2 from online and used the code linked to by the competition hosts on how to do this, tweaker the code as needed and changed the goal category, and after 10 mins of iterating it worked! And this exact method was used to also hotterdog and salt in quick succession (for salt I used the model provided FWIW, but same principle.)\n\n**SALT:** See theft.\n\n**TOKEN:** So this was clearly a well thought out and 'hard-if-you-dont-look-for-the-specific-thing' problem. I tried looking for all sorts of specific character combos by hand in excel and python but nothing I tried worked, and after a few hours of hitting dead ends, with 4 problems to go, and having not even read or attempted WAF yet, I decided to set off a barrage of notebooks brute-forcing token whilst I worked on WAF. Sure enough after many hours of hammering the API I got the answer, and saw that the killer thing to look for was 'BLANK BLANK'. Was kicking myself I didn't try to look for that. Anyway, I don't think I'd have got it without it being brute forcible (or it would have taken my WAY longer that's for sure). \n\n**CROP1**: This was nice, got it after a bit of trying, I played very specifically with the reference image and goal-seeked by hand the pixel colours etc., and simply added circles in the right places until the poisoned cropping model cropped to one of the circles I wanted it to crop to. No random walks or anything just understanding every line of the scoring function.\n\n**CROP2:** Tried a bunch of stuff, none of it worked, looking forward to seeing a working solution!\n\n**DeepFake:** The first dumb thing I tried of grabbing a video of same length worked. Which I won't lie felt good, however also realized that was not the intent of the author, and I bet it would have been super interesting and fun and rewarding to learn how to actually reverse deepfakes (or maybe next competition, to implement a deepfake? :) ) \n\n**MURDERBOTS:** This is a classic bread and butter ML problem: 'here is a test set, here is a training set, build an accurate model and prove to us it's good by submitting your predictions for the test set'. Got it using code I've used tons of times in other similar questions, but was a great fun question which will have been educational for people not used to ML or this specific concept.\n\n**Rating problem difficulties**\n\nIn terms of total difficulties, I decided the fairest metric would be how long it took me to break each problem (note - all of the problems were great and interesting and fun to crack, and this is just how long it personally took me - some questions I got hung up on just because of my own silly mistakes, others I got lucky cracking quickly):\n\nEasiest: Math 1-4, Honorstudent, Baseball, Forensics, Hotdog, Crop1, Deepfake, Murderbots\nMedium: WIFI, Theft/Salt/Hotterdog, Bad to good, WAF, Leakage\nHard: Inference, Token, Secret Sloth\nFailed to do: Crop2\n\n**Final thoughts**\n\nSo all in all - great fun, loved every second of the competition. Also credit to IsaiahP who got 21/22 flags in scarily quick time way ahead of anyone else without hints, deserving winner imho.\n\nWill see you next year hopefully but unfortunately I will likely not have as much time to sink in so I probably won't do as well from a rank perspective, but I bet I'll still have lots of fun playing around and cracking a few good puzzles.",
      "votes": null
    },
    {
      "id": "1936573",
      "postDate": "09/12/2022 21:15:42",
      "content": "<p>I loved your Kaggle Notebook and even copied some of yours Forensics snippets since I wasn't able to read the forensics hd5.</p>\n<p>Congrats for your impressive 10th place J.  Macgillivray.  Thanks for sharing your meaninful write-up.</p>",
      "rawMarkdown": "I loved your Kaggle Notebook and even copied some of yours Forensics snippets since I wasn't able to read the forensics hd5.\n\nCongrats for your impressive 10th place J.  Macgillivray.  Thanks for sharing your meaninful write-up.",
      "votes": null
    },
    {
      "id": "1937762",
      "postDate": "09/13/2022 17:27:50",
      "content": "<p>Thanks a lot! Glad it helped!</p>",
      "rawMarkdown": "Thanks a lot! Glad it helped!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1936573,
      "author_name": "mpwolke",
      "author_url": "",
      "post_date": "09/12/2022 21:15:42",
      "content": "<p>I loved your Kaggle Notebook and even copied some of yours Forensics snippets since I wasn't able to read the forensics hd5.</p>\n<p>Congrats for your impressive 10th place J.  Macgillivray.  Thanks for sharing your meaninful write-up.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1937762,
          "author_name": "johnmacgillivray",
          "author_url": "",
          "post_date": "09/13/2022 17:27:50",
          "content": "<p>Thanks a lot! Glad it helped!</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1935391": "Thankyou to Kaggle, AI Village, and my competitors for an unbelievably fun competition. I was very lucky that I was not working for 2 weeks when I found this competition, so I could become utterly obsessed and throw every waking hour at it. \n\nSolutions notebook published [here](https://www.kaggle.com/code/johnmacgillivray/894-solutions)\n\nCompetition feedback provided [here](https://www.kaggle.com/competitions/ai-village-ctf/discussion/346451#1935337) \n\n**Thoughts on each problem:**\n\n**Hotdog:** Got a picture of an actual hotdog from a dataset on kaggle for hotdog not hotdog, the ultimate classification use case.\n\n**Math 1-4:** Was already vaguely familiar with these topics so googled around to remind myself how to do it and basically followed the instructions.\n\n**WIFI:** I won't lie this one took me a depressing amount of time to get, but in the end I tried sorting the tokens by the max argument of their associated embeddings and that did the trick.\n\n**HOTTERDOG:** I used the same solution as for THEFT and SALT, see them for details.\n\n**Honorstudent:** I crudely blacked out some pixels and white pixels until the A I drew passed, very by hand, but rewarding nonetheless!\n\n**Secret Sloth:** I bashed my head against the wall for many days trying a million things, and then the competition hosts hinted at signal processing, and I tried fourier transform, but because I was trying so many things in such quick succession my attempt was sloppy in that i didnt do all the steps, and even then, i wouldnt have played with it enough to see the part of the image you needed to see. After circling back to it and trying again however, 5 lines of default inverse fourier transform being careful with the scaling got me the flag. \n\nI also found the original image on reddit and then started doing all the things that failed to work on both original image and encrypted image and looking at the deltas. This lead to a troubling set of images where I was convinced I could almost see letters. This was quite an unhinged period of the competition for me.\n\n![my suffering](https://cdn.discordapp.com/attachments/1011651849880875159/1011653051582525460/unknown.png)\n\n\n**Bad to Good:**  This one was really fun, I tried a few intuitive ideas about making Henry better and classmates worse, but to no avail. I then tried letting it do random walks just to see what would get a better score, and my naive coding led to the best walks were when Henry had a negative number of demerits. This gave me the dumb idea of 'well if its most sensitive to this and doesn't take negative amounts as illegal, lets just give Henry -20 demerits and nothing else' and it worked. Fun challenge. \n\n**Inference:**  This one was hard. I got three different datasets of written and typed characters, and started throwing them at the models to see which letters scored most highly for each of the 6 outputs. At first I was just looking for any letters which hit over a threshold, with un-flipped images. This was a bad approach in that to get any useful letter matching you have to set your threshold very high, and without flipping them you get red herrings and miss the good letters. As you will see from my solution code, what worked best for me was a climber, where to spit out a candidate image as a letter you have to be the best one yet. Another key to this was flipping the images, as was implied by the hints in the question. After some deranged playing with the candidate letters in excel, I worked out what it had to be. Took a lot of work felt good to get it right.\n\n**Baseball:**  A fun one that didn't take too long, just played around with trial and error almost by feel, working out what parameters to tweak in real time like I was playing hot/cold.\n\n**WAF:** This was great, a very educational research style puzzle. I got from the hint example string provided by hosts that this was base 64 converted and that the valid string they were getting at was 'bash'. I did not know much bash, so this lead to many hours of googling and trying stuff. I understood my goal was to write a bash command that would convince the WAF that the string I had entered was a benign string, but was struggling to implement it in a way that it actually gave me a flag rather than either triggering the WAF detection or failing to do anything cool and being a benign string. Eventually my googling led me to a specific well documented weakness that bash could exploit: 'shellshock', and code which tested exposure to that got it to work. Links in my solution code. \n\n**LEAKAGE:** I will not lie, this one exposed my lack of familiarity with LSTM and this type of model. I had to google for a few hours, and even then, I at first was stupidly just trying to go straight for the LSTM layer with my featurized vector instead of pushing it through all the layers in the model, which meant I was getting gibberish out XD. But after a day or two of staring at the boilerplate code provided I worked out what you had to do. So this was humbling and reminded me how little I know about data science, but also rewarding when I got it to work.\n\n**FORENSICS:** This was a nice quick one for me, just look at the model metadata / parameters however you want to describe it. Still educational though as it shows you how you can learn things about a model if you were trying to hack it and look for vulnerabilities, or even just trying to understand how to use it and plug it into something and make it work.\n\n**THEFT:** So this one was deceptively hard for 100 points, however as I said earlier, cracking this one instantly got me hotterdog and salt: I failed to do anything with the file provided, however by peaking at the parameters of the salt model I worked out it was a mobilnet v2. So I grabbed the pretrained mobilnet v2 from online and used the code linked to by the competition hosts on how to do this, tweaker the code as needed and changed the goal category, and after 10 mins of iterating it worked! And this exact method was used to also hotterdog and salt in quick succession (for salt I used the model provided FWIW, but same principle.)\n\n**SALT:** See theft.\n\n**TOKEN:** So this was clearly a well thought out and 'hard-if-you-dont-look-for-the-specific-thing' problem. I tried looking for all sorts of specific character combos by hand in excel and python but nothing I tried worked, and after a few hours of hitting dead ends, with 4 problems to go, and having not even read or attempted WAF yet, I decided to set off a barrage of notebooks brute-forcing token whilst I worked on WAF. Sure enough after many hours of hammering the API I got the answer, and saw that the killer thing to look for was 'BLANK BLANK'. Was kicking myself I didn't try to look for that. Anyway, I don't think I'd have got it without it being brute forcible (or it would have taken my WAY longer that's for sure). \n\n**CROP1**: This was nice, got it after a bit of trying, I played very specifically with the reference image and goal-seeked by hand the pixel colours etc., and simply added circles in the right places until the poisoned cropping model cropped to one of the circles I wanted it to crop to. No random walks or anything just understanding every line of the scoring function.\n\n**CROP2:** Tried a bunch of stuff, none of it worked, looking forward to seeing a working solution!\n\n**DeepFake:** The first dumb thing I tried of grabbing a video of same length worked. Which I won't lie felt good, however also realized that was not the intent of the author, and I bet it would have been super interesting and fun and rewarding to learn how to actually reverse deepfakes (or maybe next competition, to implement a deepfake? :) ) \n\n**MURDERBOTS:** This is a classic bread and butter ML problem: 'here is a test set, here is a training set, build an accurate model and prove to us it's good by submitting your predictions for the test set'. Got it using code I've used tons of times in other similar questions, but was a great fun question which will have been educational for people not used to ML or this specific concept.\n\n**Rating problem difficulties**\n\nIn terms of total difficulties, I decided the fairest metric would be how long it took me to break each problem (note - all of the problems were great and interesting and fun to crack, and this is just how long it personally took me - some questions I got hung up on just because of my own silly mistakes, others I got lucky cracking quickly):\n\nEasiest: Math 1-4, Honorstudent, Baseball, Forensics, Hotdog, Crop1, Deepfake, Murderbots\nMedium: WIFI, Theft/Salt/Hotterdog, Bad to good, WAF, Leakage\nHard: Inference, Token, Secret Sloth\nFailed to do: Crop2\n\n**Final thoughts**\n\nSo all in all - great fun, loved every second of the competition. Also credit to IsaiahP who got 21/22 flags in scarily quick time way ahead of anyone else without hints, deserving winner imho.\n\nWill see you next year hopefully but unfortunately I will likely not have as much time to sink in so I probably won't do as well from a rank perspective, but I bet I'll still have lots of fun playing around and cracking a few good puzzles.",
    "1936573": "I loved your Kaggle Notebook and even copied some of yours Forensics snippets since I wasn't able to read the forensics hd5.\n\nCongrats for your impressive 10th place J.  Macgillivray.  Thanks for sharing your meaninful write-up.",
    "1937762": "Thanks a lot! Glad it helped!"
  },
  "source": "meta"
}