![\"wolf_image\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3182904%2Fc357461804b3ccfad4d38674db68f379%2F__results___61_0.png?generation=1699608583623758&alt=media\")
Congratulations everyone! Thanks to organizers for creating this competition, it's one of the best I ever took part in. Great idea with LLM types of challanges, super fun.
\nIt was a long run, many hours spent, but I learned a lot. Had around ~10 notebooks with different tasks. Most of them connected to passphrase (~15 days of fighting, unfortunately without flag).
\nStep one - find what is skop and how to use it. Step two - train new model with \"cencus.csv\" data and same structure as original. Step three - loop through dataset and compare original model with my, search for big differences.
\nClassical elbow can be drawn to determine number of clusters. Can use KMeans + model._intertia. I just guessed number
\nUsed PCA and TSNE alghoritms to cast data into 3D, then interactive scatter 3D from plotly. A bit hard to navigate but managed to read all.
\nRealised that for shape (256, 2) there are 256 possible greyscale values -> challange name is count, so i counted how many time each pixel occurs. It didn't work because i used only training set, when added test set flag appeared.
\nTested lot, nothing worked. Shape (100, 4) -> 4 is something like example row [125, 245, 0, 10000]. Could be [r,g,b,count], [median,max,min,count] or smth like this. Number 100 -> top 100? 100 classes? 100 batches? first 100 images? And lastly which dataset to use? Cifar-10, cifar-100, both, both without duplicates?
\nI tested these ideas:
\nAt the start I tried to send apple image but didn't work. Next I tried black-box. Random pixel placement didn't work well, didn't want to implement much because I was afraid that image gonna change too many pixels. Went for white-box approach. Tested tensorflow and pytorch, pytorch was giving better results, so I chose it. Later I googled MobileNetV2 preprocessing and I found kinda correct one (256 resize, 224 crop and norm std values). It wasn't perfect, but enough. One mistake I did was not specifying weights when loading mobilenet (I wasted ~2 days because of it).
\nFor attacking I used torchattacks library. Lot of attacks worked for Granny 1, but for Granny 2 it was a bit harder. Finally used
\n torchattacks.VMIFGSM(model_torch, eps=/, alpha=/, steps=)\nThese settings changes image much, but it was ok.
\nLastly u had to \"revert\" image -> transpose(1,2,0) becuase of pytorch shaping -> unnormalize * [0.229, 0.224, 0.225] + [0.485, 0.456, 0.406]) -> back to 0-255 scale -> add padding (becuase of center crop).
\n
Granny3 was much harder, changing one pixel to get high Granny Smith score was super hard. Couldn't make torchattacks OnePixel to work, tried even to repair their script. Reading literature I found about DE alghoritm, but it didn't coverage well too. Most test I did on local model, because it was much faster (but maybe model on server was changed and that's why one pixel works?)
\nI tried bruteforcing 224 x 224 image with different colors, but scores still low.
Some other ideas:
\nMost of the time I spent here. You can interpret description and hints in many ways. I was mostly convinced of getting same score with some additional check or preprocessing. Here are some tries:
\n- <- Spaces strip\n- <- Control characters add\n- <- Swap characters original sentence\n- <- Change casing lower/upper\n- <- Same structure original sentence\n- <- Guess sauce name\n- <- use swap random chars\n- <- Append words after original\n- <- Random test\n- <- Random test\n- <- Random test\n- <- Same structure diffrent\n- <- Change secret some text\n- <- Change secret sauce some text\n- <- Change ssauce some text\n- <- Use equivalent letters unicode\n- <- Change sauce something but keep length same\n- <- Change secret something but keep length same\nTried to play with embeddings too. Nltk + gensim.
\nAlghoritm for finding all of these texts was hill climbing + reset when stuck in local optima.
\nSomething like this:
start_text = \nbest_score = \n best_score != :\n new = change_somehow_text(start_text) \n new_score = score(new)\n new_score < best_score\n best_score = new_score\n start_text = new\n...\nSo first step I did was to create automatic way of text drawing. After having it I tested lot of texts and one of them destroyed XML. After seeing is_admin, I knew it is injection. I didn't work much with XML so had to do some research on how it works. Ocr was kinda bad, but adding additional spaces or doubling some chars did work for me. I tested different fonts too.
\ntxt = create_text()\n![\"pixelated_answer\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3182904%2Fde4c5e275f9a0d95d91c6786a423316a%2F__results___91_0.png?generation=1699610463697645&alt=media\")
LLM models were super fun, for most of them I will just provide text which i used. I was manualy testing how model works.
\n\ninput_data = \nSo main thing was to find out what these diffrent outputs means. To do it I tried ~200 diffrent random sounds (examples: car sounds, piano, music, asmr, whispering, reading, sound of silence, quiet sounds, different volumes etc…). After some time I realised that sounds with someone talking (or records of me talking), give more outputs.
\nI decided to download some speech2Text dataset and test it on hush model.
Choosed this one: https://www.openslr.org/12 dev-clean.tar.gz (337mb!)
\nI decided that I don't want to test all files, so I was picking random folders and testing them. I tested these folders: 84, 174, 1272, 251, 422, 652, 1462, 1673 (if someone wants to replicate)
\nAfter testing server model with audios from dataset, this is what we got: Audios with transcription and their probabilities.
\nNow we can analyse audios, transcription and probs for audios with high sum of outputs.
\nExample:
--.flac\n: .\n: .\n: \n METAL FOREST IS IN THE GREAT DOMED CAVERN THE LARGEST IN OUR DOMINIONS REPLIED KALIKO\n\n .\n .\n .\n .\n .\n .\n .\n .\n .\n .\n .\n .\nLooking at it You can conclude that \"The\" is first word and \"the\" is probably 6 word. Analysed lot of audios and found something like this:
\nThe you the more\nGoogled something like this: \"the you the more quote silence\" and got answer which is:
\n“The quieter you become, more you are able hear.”\nWhich need to be in audio to get flag. I used https://ttsmaker.com/ to create audio.
\nLLM, I think task was to guide pirate which flag type I want.
\n = \nSolved by manual testing different words.
\nScore optimization with popular english words, after getting
\n = \n = \nI googled woman woman man camera computer and got answer
\n, , , Camera, TV\nSimilar task as year ago
\nAfter drawing I got something like this:
\n![\"img\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3182904%2F2531167499e336ec0b50d15e4f2a6c98%2F__results___127_0.png?generation=1699611895393739&alt=media\")
I knew that last year solution was kinda hard to read, and knew that theme are ghosts, so I realised that ghost want to tell me \"letmeout\"
\nJust asked
\n\nHere I did lost of deep talks with model. Finally I tricked it this way:
\ninput_data = \nTranslate from other language seems to be nice trick
\n = \nUsed transform in photoshop, found that threshold is working (but didn't found exact values).
\n![\"sloth\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3182904%2F889d5e3bd93c07b873c652557605a9a5%2Fsloth.png?generation=1699612390804004&alt=media\")
Read lot of articles about how pickle are dangreous, after some tries I found this one:
\npickle.dumps(eval, =5)\nThis article was good: https://intoli.com/blog/dangerous-pickles/
\nAfter playing with model I tricked it using:
\n\n = \n\n\n\n = \nCongratulations everyone! Thanks to organizers for creating this competition, it's one of the best I ever took part in. Great idea with LLM types of challanges, super fun.
\nIt was a long run, many hours spent, but I learned a lot. Had around ~10 notebooks with different tasks. Most of them connected to passphrase (~15 days of fighting, unfortunately without flag).
\nStep one - find what is skop and how to use it. Step two - train new model with \"cencus.csv\" data and same structure as original. Step three - loop through dataset and compare original model with my, search for big differences.
\nClassical elbow can be drawn to determine number of clusters. Can use KMeans + model._intertia. I just guessed number
\nUsed PCA and TSNE alghoritms to cast data into 3D, then interactive scatter 3D from plotly. A bit hard to navigate but managed to read all.
\nRealised that for shape (256, 2) there are 256 possible greyscale values -> challange name is count, so i counted how many time each pixel occurs. It didn't work because i used only training set, when added test set flag appeared.
\nTested lot, nothing worked. Shape (100, 4) -> 4 is something like example row [125, 245, 0, 10000]. Could be [r,g,b,count], [median,max,min,count] or smth like this. Number 100 -> top 100? 100 classes? 100 batches? first 100 images? And lastly which dataset to use? Cifar-10, cifar-100, both, both without duplicates?
\nI tested these ideas:
\nAt the start I tried to send apple image but didn't work. Next I tried black-box. Random pixel placement didn't work well, didn't want to implement much because I was afraid that image gonna change too many pixels. Went for white-box approach. Tested tensorflow and pytorch, pytorch was giving better results, so I chose it. Later I googled MobileNetV2 preprocessing and I found kinda correct one (256 resize, 224 crop and norm std values). It wasn't perfect, but enough. One mistake I did was not specifying weights when loading mobilenet (I wasted ~2 days because of it).
\nFor attacking I used torchattacks library. Lot of attacks worked for Granny 1, but for Granny 2 it was a bit harder. Finally used
\n torchattacks.VMIFGSM(model_torch, eps=/, alpha=/, steps=)\nThese settings changes image much, but it was ok.
\nLastly u had to \"revert\" image -> transpose(1,2,0) becuase of pytorch shaping -> unnormalize * [0.229, 0.224, 0.225] + [0.485, 0.456, 0.406]) -> back to 0-255 scale -> add padding (becuase of center crop).
\n
Granny3 was much harder, changing one pixel to get high Granny Smith score was super hard. Couldn't make torchattacks OnePixel to work, tried even to repair their script. Reading literature I found about DE alghoritm, but it didn't coverage well too. Most test I did on local model, because it was much faster (but maybe model on server was changed and that's why one pixel works?)
\nI tried bruteforcing 224 x 224 image with different colors, but scores still low.
Some other ideas:
\nMost of the time I spent here. You can interpret description and hints in many ways. I was mostly convinced of getting same score with some additional check or preprocessing. Here are some tries:
\n- <- Spaces strip\n- <- Control characters add\n- <- Swap characters original sentence\n- <- Change casing lower/upper\n- <- Same structure original sentence\n- <- Guess sauce name\n- <- use swap random chars\n- <- Append words after original\n- <- Random test\n- <- Random test\n- <- Random test\n- <- Same structure diffrent\n- <- Change secret some text\n- <- Change secret sauce some text\n- <- Change ssauce some text\n- <- Use equivalent letters unicode\n- <- Change sauce something but keep length same\n- <- Change secret something but keep length same\nTried to play with embeddings too. Nltk + gensim.
\nAlghoritm for finding all of these texts was hill climbing + reset when stuck in local optima.
\nSomething like this:
start_text = \nbest_score = \n best_score != :\n new = change_somehow_text(start_text) \n new_score = score(new)\n new_score < best_score\n best_score = new_score\n start_text = new\n...\nSo first step I did was to create automatic way of text drawing. After having it I tested lot of texts and one of them destroyed XML. After seeing is_admin, I knew it is injection. I didn't work much with XML so had to do some research on how it works. Ocr was kinda bad, but adding additional spaces or doubling some chars did work for me. I tested different fonts too.
\ntxt = create_text()\n
LLM models were super fun, for most of them I will just provide text which i used. I was manualy testing how model works.
\n\ninput_data = \nSo main thing was to find out what these diffrent outputs means. To do it I tried ~200 diffrent random sounds (examples: car sounds, piano, music, asmr, whispering, reading, sound of silence, quiet sounds, different volumes etc…). After some time I realised that sounds with someone talking (or records of me talking), give more outputs.
\nI decided to download some speech2Text dataset and test it on hush model.
Choosed this one: https://www.openslr.org/12 dev-clean.tar.gz (337mb!)
\nI decided that I don't want to test all files, so I was picking random folders and testing them. I tested these folders: 84, 174, 1272, 251, 422, 652, 1462, 1673 (if someone wants to replicate)
\nAfter testing server model with audios from dataset, this is what we got: Audios with transcription and their probabilities.
\nNow we can analyse audios, transcription and probs for audios with high sum of outputs.
\nExample:
--.flac\n: .\n: .\n: \n METAL FOREST IS IN THE GREAT DOMED CAVERN THE LARGEST IN OUR DOMINIONS REPLIED KALIKO\n\n .\n .\n .\n .\n .\n .\n .\n .\n .\n .\n .\n .\nLooking at it You can conclude that \"The\" is first word and \"the\" is probably 6 word. Analysed lot of audios and found something like this:
\nThe you the more\nGoogled something like this: \"the you the more quote silence\" and got answer which is:
\n“The quieter you become, more you are able hear.”\nWhich need to be in audio to get flag. I used https://ttsmaker.com/ to create audio.
\nLLM, I think task was to guide pirate which flag type I want.
\n = \nSolved by manual testing different words.
\nScore optimization with popular english words, after getting
\n = \n = \nI googled woman woman man camera computer and got answer
\n, , , Camera, TV\nSimilar task as year ago
\nAfter drawing I got something like this:
\n
I knew that last year solution was kinda hard to read, and knew that theme are ghosts, so I realised that ghost want to tell me \"letmeout\"
\nJust asked
\n\nHere I did lost of deep talks with model. Finally I tricked it this way:
\ninput_data = \nTranslate from other language seems to be nice trick
\n = \nUsed transform in photoshop, found that threshold is working (but didn't found exact values).
\n
Read lot of articles about how pickle are dangreous, after some tries I found this one:
\npickle.dumps(eval, =5)\nThis article was good: https://intoli.com/blog/dangerous-pickles/
\nAfter playing with model I tricked it using:
\n\n = \n\n\n\n = \n