First of all, congratulations to @icecuber. Scoring 0.79 on this competition going solo is really impressive. Also, thanks a lot to @fchollet for hosting this competition and making us face this very interesting challenge.
\n\nHere I want to briefly discuss the alrogithm that @rguigocoro and I have developed. It scores 0.87 or 0.88 on the private test set I think. The final score of 0.81 on the LB came as a result of merging teams with @yujiariyasu, who had done a great job in solving many tasks already, and we luckily had almost no overlap. His approach is explained here.
\n\nFor every task, our algorithm follows these steps:\n1. Generate an object of the class Task. At this step, all the preprocessing is done: checking for matrix shapes, common colors, symmetries... There are 6 core classes that are used to store information about the preprocessing: The classes Task, Sample, Matrix, Shape, Grid and Frontier.\n2. Based on the information retrieved from the preprocessing, the task might be transformed into something that can be processed more easily. It might make sense to modify some colors, rotate some of the matrices, crop the background or ignore a grid, for example.\n3. Once the transformation is done, we generate 3 dummy candidates and enter the loop to get the 3 best candidates. In every iteration of the loop, for each of the current best 3 candidates, we try to execute all sorts of operations that make sense to try, given the properties stored in the object of the class Task. Many different sorts of functions are tried, such as \"moveShapes\", \"replicateShapes\", \"extendColor\" or \"pixelwiseXorInGridSubmatrices\", just to mention a few examples. The complete list of functions can be found on the GitHub repository linked below. If any function generates a candidate with a better score than any of the current best 3 candidates, we include this new candidate into our list of best 3 candidates, and remove the worst one. The score is simply computed by executing the functions that led to generate that candidate in the training samples, and checking how many pixels are incorrect. Thus, a score of 0 is the best possible one.\n4. Revert the operations executed in step to 2 from the 3 candidates, to obtain the final candidates.\n5. If it makes sense, we generate another task imposing that in every sample there can be either only one shape or only one non-common color. Obviously, we then have more training and test samples than in the original task. Then, we also compute the three best candidates for this version, and compare them with the previously generated ones to select the final best candidates.
It was quite hard work to set up this whole pipeline, and the first 1.5 months or so, being unable to go under 0.99, were a bit frustrating. But in the end this has proved to work reasonably well.\nWe try lots of different functions in our algorithm, and we don't really know which ones solve tasks from the private test set, as many of our LB improvements came after general preprocessing steps that are applicable to many different kinds of tasks.
\n\nAll the code, including the version that scores 0.813 on the leaderboard, is on this GitHub repository.
", "comments": [ { "id": 865514, "author_name": "Djamel Khebab", "author_url": "", "post_date": "2020-05-28T17:12:03.413000", "content": "Thank you for sharing and congratulation.
", "votes": 1, "replies": [] }, { "id": 864852, "author_name": "Jan Bre", "author_url": "", "post_date": "2020-05-28T08:22:58.557000", "content": "Congratulations on that stunning result
", "votes": 1, "replies": [] }, { "id": 864779, "author_name": "YujiAriyasu", "author_url": "", "post_date": "2020-05-28T07:21:46.980000", "content": "I am so proud of you, my friend.
", "votes": 2, "replies": [] }, { "id": 1280379, "author_name": "Philip Dhingra", "author_url": "", "post_date": "2021-04-21T21:12:37.990000", "content": "I noticed that on the GitHub, it says that the program has been tested on iOS! I'm curious now what that means. Were you entering the Kaggle competition from your iPhone??
", "votes": 0, "replies": [ { "id": 1280399, "author_name": "Alejandro de Miquel", "author_url": "", "post_date": "2021-04-21T22:04:20.520000", "content": "Well you're right, it should say Mac OS. Thanks for pointing this out.
", "votes": 1, "replies": [] } ] }, { "id": 865197, "author_name": "Froggy McFrogson", "author_url": "", "post_date": "2020-05-28T13:13:35.060000", "content": "What happens if you get 4 answers with a score of 0? I suggest that then you would order them by \"simplest\" transformations first.
\n\nWell done on 2nd place. Well deserved.
", "votes": 0, "replies": [ { "id": 865806, "author_name": "Alejandro de Miquel", "author_url": "", "post_date": "2020-05-28T22:39:49.033000", "content": "Yes, whenever there is a tie on the score, we choose the simplest transformation. By simplest, we mean the one that requires less operations.
", "votes": 0, "replies": [] } ] }, { "id": 872097, "author_name": "corochann", "author_url": "", "post_date": "2020-06-02T23:04:40.393000", "content": "Conglats! thanks for sharing.
", "votes": 0, "replies": [] } ], "comment_count": 8, "messages": [ { "id": 864768, "postDate": "2020-05-28T07:12:19.760Z", "content": "First of all, congratulations to @icecuber. Scoring 0.79 on this competition going solo is really impressive. Also, thanks a lot to @fchollet for hosting this competition and making us face this very interesting challenge.
\n\nHere I want to briefly discuss the alrogithm that @rguigocoro and I have developed. It scores 0.87 or 0.88 on the private test set I think. The final score of 0.81 on the LB came as a result of merging teams with @yujiariyasu, who had done a great job in solving many tasks already, and we luckily had almost no overlap. His approach is explained here.
\n\nFor every task, our algorithm follows these steps:\n1. Generate an object of the class Task. At this step, all the preprocessing is done: checking for matrix shapes, common colors, symmetries... There are 6 core classes that are used to store information about the preprocessing: The classes Task, Sample, Matrix, Shape, Grid and Frontier.\n2. Based on the information retrieved from the preprocessing, the task might be transformed into something that can be processed more easily. It might make sense to modify some colors, rotate some of the matrices, crop the background or ignore a grid, for example.\n3. Once the transformation is done, we generate 3 dummy candidates and enter the loop to get the 3 best candidates. In every iteration of the loop, for each of the current best 3 candidates, we try to execute all sorts of operations that make sense to try, given the properties stored in the object of the class Task. Many different sorts of functions are tried, such as \"moveShapes\", \"replicateShapes\", \"extendColor\" or \"pixelwiseXorInGridSubmatrices\", just to mention a few examples. The complete list of functions can be found on the GitHub repository linked below. If any function generates a candidate with a better score than any of the current best 3 candidates, we include this new candidate into our list of best 3 candidates, and remove the worst one. The score is simply computed by executing the functions that led to generate that candidate in the training samples, and checking how many pixels are incorrect. Thus, a score of 0 is the best possible one.\n4. Revert the operations executed in step to 2 from the 3 candidates, to obtain the final candidates.\n5. If it makes sense, we generate another task imposing that in every sample there can be either only one shape or only one non-common color. Obviously, we then have more training and test samples than in the original task. Then, we also compute the three best candidates for this version, and compare them with the previously generated ones to select the final best candidates.
It was quite hard work to set up this whole pipeline, and the first 1.5 months or so, being unable to go under 0.99, were a bit frustrating. But in the end this has proved to work reasonably well.\nWe try lots of different functions in our algorithm, and we don't really know which ones solve tasks from the private test set, as many of our LB improvements came after general preprocessing steps that are applicable to many different kinds of tasks.
\n\nAll the code, including the version that scores 0.813 on the leaderboard, is on this GitHub repository.
", "rawMarkdown": "First of all, congratulations to @icecuber. Scoring 0.79 on this competition going solo is really impressive. Also, thanks a lot to @fchollet for hosting this competition and making us face this very interesting challenge.\n\nHere I want to briefly discuss the alrogithm that @rguigocoro and I have developed. It scores 0.87 or 0.88 on the private test set I think. The final score of 0.81 on the LB came as a result of merging teams with @yujiariyasu, who had done a great job in solving many tasks already, and we luckily had almost no overlap. His approach is explained [here](https://www.kaggle.com/c/abstraction-and-reasoning-challenge/discussion/138548?fbclid=IwAR0rVWjYYPzF3hDFT109Jsp-ezHsqttJzsVBgnVyG4JrFRUH7tBPIlia4y0).\n\n\nFor every task, our algorithm follows these steps:\n1. Generate an object of the class `Task`. At this step, all the preprocessing is done: checking for matrix shapes, common colors, symmetries... There are 6 core classes that are used to store information about the preprocessing: The classes `Task`, `Sample`, `Matrix`, `Shape`, `Grid` and `Frontier`.\n2. Based on the information retrieved from the preprocessing, the task might be transformed into something that can be processed more easily. It might make sense to modify some colors, rotate some of the matrices, crop the background or ignore a grid, for example.\n3. Once the transformation is done, we generate 3 dummy candidates and enter the loop to get the 3 best candidates. In every iteration of the loop, for each of the current best 3 candidates, we try to execute all sorts of operations that make sense to try, given the properties stored in the object of the class `Task`. Many different sorts of functions are tried, such as \"moveShapes\", \"replicateShapes\", \"extendColor\" or \"pixelwiseXorInGridSubmatrices\", just to mention a few examples. The complete list of functions can be found on the GitHub repository linked below. If any function generates a candidate with a better score than any of the current best 3 candidates, we include this new candidate into our list of best 3 candidates, and remove the worst one. The score is simply computed by executing the functions that led to generate that candidate in the training samples, and checking how many pixels are incorrect. Thus, a score of 0 is the best possible one.\n4. Revert the operations executed in step to 2 from the 3 candidates, to obtain the final candidates.\n5. If it makes sense, we generate another task imposing that in every sample there can be either only one shape or only one non-common color. Obviously, we then have more training and test samples than in the original task. Then, we also compute the three best candidates for this version, and compare them with the previously generated ones to select the final best candidates.\n\nIt was quite hard work to set up this whole pipeline, and the first 1.5 months or so, being unable to go under 0.99, were a bit frustrating. But in the end this has proved to work reasonably well.\nWe try lots of different functions in our algorithm, and we don't really know which ones solve tasks from the private test set, as many of our LB improvements came after general preprocessing steps that are applicable to many different kinds of tasks.\n\nAll the code, including the version that scores 0.813 on the leaderboard, is on [this](https://github.com/alejandrodemiquel/ARC_Kaggle) GitHub repository.", "votes": 39 }, { "id": 865514, "postDate": "2020-05-28T17:12:03.413Z", "content": "Thank you for sharing and congratulation.
", "rawMarkdown": "Thank you for sharing and congratulation.", "votes": 1 }, { "id": 864852, "postDate": "2020-05-28T08:22:58.557Z", "content": "Congratulations on that stunning result
", "rawMarkdown": "Congratulations on that stunning result", "votes": 1 }, { "id": 864779, "postDate": "2020-05-28T07:21:46.980Z", "content": "I am so proud of you, my friend.
", "rawMarkdown": "I am so proud of you, my friend.", "votes": 2 }, { "id": 1280379, "postDate": "2021-04-21T21:12:37.990Z", "content": "I noticed that on the GitHub, it says that the program has been tested on iOS! I'm curious now what that means. Were you entering the Kaggle competition from your iPhone??
", "rawMarkdown": "I noticed that on the GitHub, it says that the program has been tested on iOS! I'm curious now what that means. Were you entering the Kaggle competition from your iPhone??", "replies": [ { "id": 1280399, "postDate": "2021-04-21T22:04:20.520Z", "content": "Well you're right, it should say Mac OS. Thanks for pointing this out.
", "rawMarkdown": "Well you're right, it should say Mac OS. Thanks for pointing this out.", "votes": 1 } ] }, { "id": 865197, "postDate": "2020-05-28T13:13:35.060Z", "content": "What happens if you get 4 answers with a score of 0? I suggest that then you would order them by \"simplest\" transformations first.
\n\nWell done on 2nd place. Well deserved.
", "rawMarkdown": "What happens if you get 4 answers with a score of 0? I suggest that then you would order them by \"simplest\" transformations first.\n\nWell done on 2nd place. Well deserved.", "replies": [ { "id": 865806, "postDate": "2020-05-28T22:39:49.033Z", "content": "Yes, whenever there is a tie on the score, we choose the simplest transformation. By simplest, we mean the one that requires less operations.
", "rawMarkdown": "Yes, whenever there is a tie on the score, we choose the simplest transformation. By simplest, we mean the one that requires less operations." } ] }, { "id": 872097, "postDate": "2020-06-02T23:04:40.393Z", "content": "Conglats! thanks for sharing.
", "rawMarkdown": "Conglats! thanks for sharing." } ], "complete": true, "source": "live", "fetched_at": "2026-10-19T22:06:25Z", "competition_id": 18329, "competition_slug": "abstraction-and-reasoning-challenge", "team_id": 4479414, "solution_url": "/competitions/abstraction-and-reasoning-challenge/writeups/alejandro-roderic-yuji-2nd-place-solution" }