Congratulations to everyone that the competition is finally over! Just after the end I would like to tell you a bit about the solution of our team “Puzzlemaster” that got 10th place.
\n\nOur solution is an ensemble of 3 parts that solved 3 sets of tasks that has a partial intersection:\n- DSL-algorithm implemented by @felipefonte99 \n- ML-algorithm implemented by @nikitaovsov. \nI think they will create other posts about their solutions or post comments in this thread.
\n\nAt first I would like to thank authors of these kernels that inspired me from the algorithm’s perspective:
\n\nhttps://www.kaggle.com/arsenynerinovsky/cellular-automata-as-a-language-for-reasoning\nhttps://www.kaggle.com/zenol42/dsl-and-genetic-algorithm-applied-to-arc
\n\nAnd of course the kernel that I opened 1000 times:
\n\nhttps://www.kaggle.com/boliu0/visualizing-all-task-pairs-with-gridlines
\n\nThe solution
\n\nWe created a Domain-Specific Language based on commands that can be applied to a grid.\nThere are many commands in our language. Each command has a couple of parameters that are randomly sampled during training. We call it “global rules” (in opposite to cellular automata rules that are described below)\nExamples of the commands:\n- Draw a line (or some color) from cells (of some color) (until it stops at some color).\n- Map all cells of (some color) to (some color)\n- Rotate the grid\n- Apply gravity to (figures/cells) of (some color) in the direction of ((some border)/some color)\n- others\nAll that written in brackets are randomly sampled parameters.
\n\nAt the end of the competition we added elements of “Multiple grids” to the DSL. For example, it could split the grid (for example, by figures on it), apply some commands to some of the grids and then merge them back. This boosted our LB score a lot.
\n\nThere were also “Cellular automata” commands that were applied after previous commands. More details about cellular automatas can be find at this kernel:\nhttps://www.kaggle.com/arsenynerinovsky/cellular-automata-as-a-language-for-reasoning
\n\nOur DSL contained a couple of randomly parametrized CA-rules. For example, checking of the direct neighbors, checking of colored corners and so on.
\n\nSo the final DSL-program is a sequence of “global” rules, then CA-rules are applied to the result of that.
\n\nThe C++ part\nVery important part of our solution was that the “global” rules and “CA-rules” were implemented by @artyomp using C++. He did a great job that accelerated the whole algorithm at 100 times and allowed us to create many new rules and not to care much about the complexity of the search space. Without C++, we would probably solve half of the test tasks we solved.
\n\nThe search algorithm\nIn fact, it’s similar to the algorithm from https://www.kaggle.com/zenol42/dsl-and-genetic-algorithm-applied-to-arc but with a number of heuristics.\nOur algorithm creates random sequences of programs, creates asexual mutations (tries to replace commands with new commands, insert new commands, delete commands) and \"sexual\" mutations (tries to concatenate random prefixes and suffixes of 2 programs with some mutation in the end).
\n\nQuality measurement\nHow to measure if the program is good? Obviously, we should measure accuracy (how many cells are correct). If it equals to 1 for all training pairs, we found a solution. But since we use genetic algorithm with mutations of 1 rule to find a solution, we might need more metrics to allow the population to be bigger. My observation was that we should be careful with that because if we add more metrics, it is more likely that a junky program with many commands will be created by the algorithm that can solve training samples but overfits.
\n\nSo as a result, our final choice of metrics:\n- Mean accuracy for all training outputs\n- Accuracies for each output\n- The mean of the following binary metrics: for each color from 0 to 10 we compute if our prediction of the color is absolutely correct (in terms of both precision and recall). It helped the algorithm to find correct sequences that fit correctly colors one by one.
\n\nThe code\nThe DSL-solution is about 2.5k lines of code. We are going to clean it a little bit, make some comments and publish as a kernel so you can dive into details.\nI didn’t mention a huge number of details of the algorithm because there would be too much text. But I am ready to answer your questions about the algorithms if you are curious.
\n\nUPD.
\n\nThe C++ version is in repo:\nhttps://github.com/artyompal/kaggle-abstract-reasoning
\n\nThe kernel with the python version:\nhttps://www.kaggle.com/alexfritz/genetic-dsl-part-from-10th-place-python-version
\n\nConclusion\nI would like to thank organizers and my team @artyomp @nikitaovsov @felipefonte99! It was a great experience for me and I learned a lot of new things.
", "comments": [ { "id": 865131, "author_name": "Artyom Palvelev", "author_url": "", "post_date": "2020-05-28T12:14:57.160000", "content": "My story here. Ok, last 3 months were painful. I created 4 GitHub repositories, so I tried this many approaches.
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1859557%2Feec1ca242c646a41f75ea22edbeae004%2F0d62c1b.png?generation=1590666024458460&alt=media\")
The program it found was (reverse Polish notation, like OPCODE OP1 OP2):\nif and eq border 0 eq color 0 0 0 paint 4\nrepeat 1 if and eq color any_von_neimann_nei 0 eq color 0 0 4 paint 0\nfrom dsl import cpp_trace_param_automata\n...\noutput = cpp_trace_param_automata(input, params, n_iter) # input and output are is numpy arrays\n\nIn C++ code, you write:\n```\nnamespace py = pybind11;
\n\npy::array_t<ssize_t> cpp_trace_param_automata(py::array_t<ssize_t> input, py::list params_list, int n_iter) {\n...\n}
\n\nPYBIND11_MODULE(dsl, m) {\n m.def(\"cpp_trace_param_automata\", &cpp_trace_param_automata,\n \"A function which simulates an automaton\");\n}\n``\nThen you have to copy-paste a workingsetup.pyand just dopip install .` to build it.
So, this improved performance by orders of magnitude. We could try a lot more things now. We came up with a feature-rich DSL. Not only it used Cellular Automata, but it also could move shapes, match them, paint lines and much, much more! We think that it solves about 10 tasks from the test set.
", "votes": 7, "replies": [ { "id": 865138, "author_name": "Artyom Palvelev", "author_url": "", "post_date": "2020-05-28T12:22:35.533000", "content": "An example of pybind11 interface usage:\n```\n// Parameters: a custom class and a list of List[Dict], List[Dict], Dict and Dict.\nvoid convert_all_rules(AutomatonParams ¶ms, const py::list ¶ms_list) {\n assert(params_list.size() == 4);
\n\nfor (const auto &item : params_list[0].cast<py::list>()) {\n params.global_rules.push_back(convert_rule(item.cast<py::dict>()));\n}\n\nfor (const auto &item : params_list[1].cast<py::list>()) {\n params.ca_rules.push_back(convert_rule(item.cast<py::dict>()));\n}\n\n// We can even debug print whole objects using their __str__ method!\n// puts(std::string(params_list[2].cast<py::str>()).c_str());\n\nparams.split_rule = convert_rule(params_list[2].cast<py::dict>());\nparams.merge_rule = convert_rule(params_list[3].cast<py::dict>());\n}
\n\n// This receives np.array and converts it into a custom format\nvoid convert_automaton_state(AutomatonState &state, const py::array_t<ssize_t> &input) {\n auto buf = input.request();\n char *ptr = (char *) buf.ptr;\n ssize_t stride0 = input.strides(0), stride1 = input.strides(1);
\n\nfor (uint i = 0; i < state.shape[0]; i++) {\n for (uint j = 0; j < state.shape[1]; j++) {\n state(i, j) = (ssize_t &) ptr[i * stride0 + j * stride1];\n }\n}\n}
\n\n// Here we create a np.array in C++ and to return it into Python. Works like a charm!\npy::array_t<ssize_t> convert_result(const AutomatonState &state) {\n py::array_t<ssize_t> output(state.color.size());\n auto buf = output.request();\n ssize_t *ptr = (ssize_t *) buf.ptr;
\n\nfor (uint i = 0; i < state.shape[0]; i++) {\n for (uint j = 0; j < state.shape[1]; j++) {\n ptr[i * state.shape[1] + j] = state(i, j);\n }\n}\n\noutput.resize({state.shape[0], state.shape[1]});\n// printf(\"output shape: %d x %d\\n\", (int) output.shape(0), (int) output.shape(1));\nreturn output;\n}\n```
", "votes": 2, "replies": [] }, { "id": 865328, "author_name": "", "author_url": "", "post_date": "2020-05-28T14:57:16.557000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 864722, "author_name": "Ovsov Nikita", "author_url": "", "post_date": "2020-05-28T06:45:48.493000", "content": "A couple of words about my part of the solution mentioned in the post.\nI tried to solve puzzles, when a solution is the part (crop) of the puzzle itself. For each task I check, whether solutions are crops or not. If it is, I train a simple ML model on all possible crops. \nIt gave me 0.97 on the leaderboard, and 0.95 after ensembling with two public kernels.
\n\nDuring the last week of the competition, after teaming up with these great guys, I added the ability not only to cut crops from the input image, but also change colors.\nSo my part of the solution could possibly solve tasks, where the answer is a combination of cutting crop and color repainting, or just color repainting (in this case crop is equal to image itself).\nIt gave me 6 more tasks on the validation set, but unfortunately it didn't improve our public score.
\n\nI would also like to thank my team @alexfritz @artyomp @felipefonte99 ! You did a great job!
", "votes": 3, "replies": [] }, { "id": 864319, "author_name": "Felipe Loque", "author_url": "", "post_date": "2020-05-28T00:19:17.957000", "content": "Here is my part mentioned above:
\n\nDescription\nMy DSL is based on 'operations'. It applies a lot of operations on the input and stores the result (prediction) of each operation. For example: crop the biggest object, crop the object of the less frequent color, flip some object, change background color, etc. etc. And also those operations combined. So it tries thousands of operations.
\n\nLoss\nWith thousands of predictions, which one should I choose? It's simple: for each prediction I calculate a loss based on how many parts of the training set of that task it solved. For example, if a loss for a specific prediction is 0, it means that operation solved all parts of the training set, so it must be an excellent prediction for the test part. So in the end I just select the 3 predictions with the smallest loss.
\n\nResult\nThis algorithm solved 5 tasks of the test set (score 0.95), all of them were tasks where the output size is smaller than the input. I also ensembled with two public kernels to solve 2 more tasks and get 0.93 on the leaderboard before teaming up with my great teammates.
", "votes": 4, "replies": [ { "id": 864426, "author_name": "higepon", "author_url": "", "post_date": "2020-05-28T02:02:39.733000", "content": "Thanks for sharing. The Loss part is super interesting. wow.
", "votes": 1, "replies": [] } ] }, { "id": 864322, "author_name": "Froggy McFrogson", "author_url": "", "post_date": "2020-05-28T00:19:36.913000", "content": "That's very interesting. In your DSL language how does it cope with this like \"change all the shapes with the highest number of holes in them to green\" ?
\n\nThe way I tried to do it was look at the inputs and outputs, and try to find out what were the changes and what properties of the shapes that changed and then try to work out a rule.
\n\nAs well as trying lots of different brute force things too like symmetries etc.
", "votes": 0, "replies": [ { "id": 865184, "author_name": "Artyom Palvelev", "author_url": "", "post_date": "2020-05-28T13:02:25.913000", "content": "I don't think there are tasks in the test set like this. My XGBoost approach could count holes by counting the number of connected components. This solved many tasks on train and validation, but didn't improve the score.
\n\nWe have to think in a more general way.
", "votes": 0, "replies": [] } ] }, { "id": 866635, "author_name": "", "author_url": "", "post_date": "2020-05-29T15:02:02.150000", "content": "", "votes": 0, "replies": [] }, { "id": 872107, "author_name": "corochann", "author_url": "", "post_date": "2020-06-02T23:09:58.560000", "content": "Conglats! thanks for sharing.
", "votes": 1, "replies": [] } ], "comment_count": 10, "messages": [ { "id": 864311, "postDate": "2020-05-28T00:13:37.897Z", "content": "Congratulations to everyone that the competition is finally over! Just after the end I would like to tell you a bit about the solution of our team “Puzzlemaster” that got 10th place.
\n\nOur solution is an ensemble of 3 parts that solved 3 sets of tasks that has a partial intersection:\n- DSL-algorithm implemented by @felipefonte99 \n- ML-algorithm implemented by @nikitaovsov. \nI think they will create other posts about their solutions or post comments in this thread.
\n\nAt first I would like to thank authors of these kernels that inspired me from the algorithm’s perspective:
\n\nhttps://www.kaggle.com/arsenynerinovsky/cellular-automata-as-a-language-for-reasoning\nhttps://www.kaggle.com/zenol42/dsl-and-genetic-algorithm-applied-to-arc
\n\nAnd of course the kernel that I opened 1000 times:
\n\nhttps://www.kaggle.com/boliu0/visualizing-all-task-pairs-with-gridlines
\n\nThe solution
\n\nWe created a Domain-Specific Language based on commands that can be applied to a grid.\nThere are many commands in our language. Each command has a couple of parameters that are randomly sampled during training. We call it “global rules” (in opposite to cellular automata rules that are described below)\nExamples of the commands:\n- Draw a line (or some color) from cells (of some color) (until it stops at some color).\n- Map all cells of (some color) to (some color)\n- Rotate the grid\n- Apply gravity to (figures/cells) of (some color) in the direction of ((some border)/some color)\n- others\nAll that written in brackets are randomly sampled parameters.
\n\nAt the end of the competition we added elements of “Multiple grids” to the DSL. For example, it could split the grid (for example, by figures on it), apply some commands to some of the grids and then merge them back. This boosted our LB score a lot.
\n\nThere were also “Cellular automata” commands that were applied after previous commands. More details about cellular automatas can be find at this kernel:\nhttps://www.kaggle.com/arsenynerinovsky/cellular-automata-as-a-language-for-reasoning
\n\nOur DSL contained a couple of randomly parametrized CA-rules. For example, checking of the direct neighbors, checking of colored corners and so on.
\n\nSo the final DSL-program is a sequence of “global” rules, then CA-rules are applied to the result of that.
\n\nThe C++ part\nVery important part of our solution was that the “global” rules and “CA-rules” were implemented by @artyomp using C++. He did a great job that accelerated the whole algorithm at 100 times and allowed us to create many new rules and not to care much about the complexity of the search space. Without C++, we would probably solve half of the test tasks we solved.
\n\nThe search algorithm\nIn fact, it’s similar to the algorithm from https://www.kaggle.com/zenol42/dsl-and-genetic-algorithm-applied-to-arc but with a number of heuristics.\nOur algorithm creates random sequences of programs, creates asexual mutations (tries to replace commands with new commands, insert new commands, delete commands) and \"sexual\" mutations (tries to concatenate random prefixes and suffixes of 2 programs with some mutation in the end).
\n\nQuality measurement\nHow to measure if the program is good? Obviously, we should measure accuracy (how many cells are correct). If it equals to 1 for all training pairs, we found a solution. But since we use genetic algorithm with mutations of 1 rule to find a solution, we might need more metrics to allow the population to be bigger. My observation was that we should be careful with that because if we add more metrics, it is more likely that a junky program with many commands will be created by the algorithm that can solve training samples but overfits.
\n\nSo as a result, our final choice of metrics:\n- Mean accuracy for all training outputs\n- Accuracies for each output\n- The mean of the following binary metrics: for each color from 0 to 10 we compute if our prediction of the color is absolutely correct (in terms of both precision and recall). It helped the algorithm to find correct sequences that fit correctly colors one by one.
\n\nThe code\nThe DSL-solution is about 2.5k lines of code. We are going to clean it a little bit, make some comments and publish as a kernel so you can dive into details.\nI didn’t mention a huge number of details of the algorithm because there would be too much text. But I am ready to answer your questions about the algorithms if you are curious.
\n\nUPD.
\n\nThe C++ version is in repo:\nhttps://github.com/artyompal/kaggle-abstract-reasoning
\n\nThe kernel with the python version:\nhttps://www.kaggle.com/alexfritz/genetic-dsl-part-from-10th-place-python-version
\n\nConclusion\nI would like to thank organizers and my team @artyomp @nikitaovsov @felipefonte99! It was a great experience for me and I learned a lot of new things.
", "rawMarkdown": "Congratulations to everyone that the competition is finally over! Just after the end I would like to tell you a bit about the solution of our team “Puzzlemaster” that got 10th place.\n\nOur solution is an ensemble of 3 parts that solved 3 sets of tasks that has a partial intersection:\n- DSL-algorithm implemented by @felipefonte99 \n- ML-algorithm implemented by @nikitaovsov. \nI think they will create other posts about their solutions or post comments in this thread.\n\n- Genetic algorithm that tries to fit a DSL-program that was created by me and @artyomp. It solved 10-12 tasks. \nNow I am going to tell you about it.\n\nAt first I would like to thank authors of these kernels that inspired me from the algorithm’s perspective:\n\n[https://www.kaggle.com/arsenynerinovsky/cellular-automata-as-a-language-for-reasoning](https://www.kaggle.com/arsenynerinovsky/cellular-automata-as-a-language-for-reasoning)\n[https://www.kaggle.com/zenol42/dsl-and-genetic-algorithm-applied-to-arc](https://www.kaggle.com/zenol42/dsl-and-genetic-algorithm-applied-to-arc)\n\nAnd of course the kernel that I opened 1000 times:\n\n[https://www.kaggle.com/boliu0/visualizing-all-task-pairs-with-gridlines](https://www.kaggle.com/boliu0/visualizing-all-task-pairs-with-gridlines)\n\n**The solution**\n\nWe created a Domain-Specific Language based on commands that can be applied to a grid.\nThere are many commands in our language. Each command has a couple of parameters that are randomly sampled during training. We call it “global rules” (in opposite to cellular automata rules that are described below)\nExamples of the commands:\n- Draw a line (or some color) from cells (of some color) (until it stops at some color).\n- Map all cells of (some color) to (some color)\n- Rotate the grid\n- Apply gravity to (figures/cells) of (some color) in the direction of ((some border)/some color)\n- others\nAll that written in brackets are randomly sampled parameters.\n\nAt the end of the competition we added elements of “Multiple grids” to the DSL. For example, it could split the grid (for example, by figures on it), apply some commands to some of the grids and then merge them back. This boosted our LB score a lot.\n\nThere were also “Cellular automata” commands that were applied after previous commands. More details about cellular automatas can be find at this kernel:\nhttps://www.kaggle.com/arsenynerinovsky/cellular-automata-as-a-language-for-reasoning\n\nOur DSL contained a couple of randomly parametrized CA-rules. For example, checking of the direct neighbors, checking of colored corners and so on.\n\nSo the final DSL-program is a sequence of “global” rules, then CA-rules are applied to the result of that.\n\n**The C++ part**\nVery important part of our solution was that the “global” rules and “CA-rules” were implemented by @artyomp using C++. He did a great job that accelerated the whole algorithm at 100 times and allowed us to create many new rules and not to care much about the complexity of the search space. Without C++, we would probably solve half of the test tasks we solved.\n\n\n**The search algorithm**\nIn fact, it’s similar to the algorithm from https://www.kaggle.com/zenol42/dsl-and-genetic-algorithm-applied-to-arc but with a number of heuristics.\nOur algorithm creates random sequences of programs, creates asexual mutations (tries to replace commands with new commands, insert new commands, delete commands) and \"sexual\" mutations (tries to concatenate random prefixes and suffixes of 2 programs with some mutation in the end).\n\n**Quality measurement**\nHow to measure if the program is good? Obviously, we should measure accuracy (how many cells are correct). If it equals to 1 for all training pairs, we found a solution. But since we use genetic algorithm with mutations of 1 rule to find a solution, we might need more metrics to allow the population to be bigger. My observation was that we should be careful with that because if we add more metrics, it is more likely that a junky program with many commands will be created by the algorithm that can solve training samples but overfits. \n\nSo as a result, our final choice of metrics:\n- Mean accuracy for all training outputs\n- Accuracies for each output\n- The mean of the following binary metrics: for each color from 0 to 10 we compute if our prediction of the color is absolutely correct (in terms of both precision and recall). It helped the algorithm to find correct sequences that fit correctly colors one by one.\n\n**The code**\nThe DSL-solution is about 2.5k lines of code. We are going to clean it a little bit, make some comments and publish as a kernel so you can dive into details.\nI didn’t mention a huge number of details of the algorithm because there would be too much text. But I am ready to answer your questions about the algorithms if you are curious.\n\nUPD.\n\nThe C++ version is in repo:\nhttps://github.com/artyompal/kaggle-abstract-reasoning\n\nThe kernel with the python version:\nhttps://www.kaggle.com/alexfritz/genetic-dsl-part-from-10th-place-python-version\n\n**Conclusion**\nI would like to thank organizers and my team @artyomp @nikitaovsov @felipefonte99! It was a great experience for me and I learned a lot of new things.\n", "votes": 34 }, { "id": 865131, "postDate": "2020-05-28T12:14:57.160Z", "content": "My story here. Ok, last 3 months were painful. I created 4 GitHub repositories, so I tried this many approaches.
\n\n The program it found was (reverse Polish notation, like OPCODE OP1 OP2):\nif and eq border 0 eq color 0 0 0 paint 4\nrepeat 1 if and eq color any_von_neimann_nei 0 eq color 0 0 4 paint 0\nfrom dsl import cpp_trace_param_automata\n...\noutput = cpp_trace_param_automata(input, params, n_iter) # input and output are is numpy arrays\n\nIn C++ code, you write:\n```\nnamespace py = pybind11;
\n\npy::array_t<ssize_t> cpp_trace_param_automata(py::array_t<ssize_t> input, py::list params_list, int n_iter) {\n...\n}
\n\nPYBIND11_MODULE(dsl, m) {\n m.def(\"cpp_trace_param_automata\", &cpp_trace_param_automata,\n \"A function which simulates an automaton\");\n}\n``\nThen you have to copy-paste a workingsetup.pyand just dopip install .` to build it.
So, this improved performance by orders of magnitude. We could try a lot more things now. We came up with a feature-rich DSL. Not only it used Cellular Automata, but it also could move shapes, match them, paint lines and much, much more! We think that it solves about 10 tasks from the test set.
", "rawMarkdown": "My story here. Ok, last 3 months were painful. I created 4 GitHub repositories, so I tried this many approaches.\n\n1. I started in the late February. In the next 3 weeks I played with a genetic algorithm in Python but it wouldn't solve anything difficult. Also I tried Recurrent CNN (https://www.kaggle.com/teddykoker/training-cellular-automata-part-ii-learning-tasks), but it would not generalize to any new samples.\n2. I decided to implement a Cellular Automata solution in C++. In the next 3 weeks I made a simple brute-force algorithm, but it was very efficient. It could check out 100k to 1M cellular automata per second, so it solved 17 tasks from the train and 2 tasks from the validation dataset. It could solve some simple tasks like 00d62c1b:  The program it found was (reverse Polish notation, like OPCODE OP1 OP2):\n```if and eq border 0 eq color 0 0 0 paint 4 ```\n```repeat 1 if and eq color any_von_neimann_nei 0 eq color 0 0 4 paint 0 ```\n3. Then I tried to make my genetic algorithm find this programs, and it didn't work well. It was mid-April, I was getting a bit desperate. A solution with BERT (https://www.kaggle.com/alvaroma/bert-abstraction) impressed me a lot, but I decided that it just overfits. So I decided to play with XGBoost approach (0.99 LB). I took a baseline and started adding features. First thing I noticed was, the background in different samples might be different. So, I made a frequency encoder: replace the most frequent color with 0, next will be 1, and so on. This solved one more sample from the test (0.98). For some tasks it helped, so I solved every task with and without it. I added KFold validation (sort of Leave One Out scheme, in terms of train samples of every task) and made it pick top-3 predictions by validation accuracy. I played with feature engineering, but it was mostly helping on train/validation only. One simple feature solved one more sample though: adding i % 2, j % 2 features. It solves the same sample that ModuleQ notebook is solving. So finally, this approach solved 116 tasks from the train, 65 tasks from the validation and 4 tasks from the test set.\n4. We were talking to @alexfritz about a possible merge. I saw potential in his solution and suggested that it could be improved with C++. So we made a team. I threw my XGBoost solution to the bin and started porting his Cellular Automata code to C++ (reusing my system from part 2). So, last 2 weeks were painful, but we made it. Initially, I couldn't find an efficient way of connecting C++ code to Python, so I opened a socket in C++ code and started sending JSON-encoded requests from Python. Then a friend told me about pybind11. pybind11 is actually a great library, maybe even a bit oversimplified (I don't like it's automatic conversions because of the performance hit). It even support NumPy arrays out of box. In Python code, you write:\n```\nfrom dsl import cpp_trace_param_automata\n...\noutput = cpp_trace_param_automata(input, params, n_iter) # input and output are is numpy arrays\n```\nIn C++ code, you write:\n```\n#include <pybind11/pybind11.h>\n#include <pybind11/numpy.h>\n\nnamespace py = pybind11;\n\npy::array_t<ssize_t> cpp_trace_param_automata(py::array_t<ssize_t> input, py::list params_list, int n_iter) {\n...\n}\n\nPYBIND11_MODULE(dsl, m) {\n m.def(\"cpp_trace_param_automata\", &cpp_trace_param_automata,\n \"A function which simulates an automaton\");\n}\n```\nThen you have to copy-paste a working `setup.py` and just do `pip install .` to build it.\n\nSo, this improved performance by orders of magnitude. We could try a lot more things now. We came up with a feature-rich DSL. Not only it used Cellular Automata, but it also could move shapes, match them, paint lines and much, much more! We think that it solves about 10 tasks from the test set.", "votes": 7, "replies": [ { "id": 865138, "postDate": "2020-05-28T12:22:35.533Z", "content": "An example of pybind11 interface usage:\n```\n// Parameters: a custom class and a list of List[Dict], List[Dict], Dict and Dict.\nvoid convert_all_rules(AutomatonParams ¶ms, const py::list ¶ms_list) {\n assert(params_list.size() == 4);
\n\nfor (const auto &item : params_list[0].cast<py::list>()) {\n params.global_rules.push_back(convert_rule(item.cast<py::dict>()));\n}\n\nfor (const auto &item : params_list[1].cast<py::list>()) {\n params.ca_rules.push_back(convert_rule(item.cast<py::dict>()));\n}\n\n// We can even debug print whole objects using their __str__ method!\n// puts(std::string(params_list[2].cast<py::str>()).c_str());\n\nparams.split_rule = convert_rule(params_list[2].cast<py::dict>());\nparams.merge_rule = convert_rule(params_list[3].cast<py::dict>());\n}
\n\n// This receives np.array and converts it into a custom format\nvoid convert_automaton_state(AutomatonState &state, const py::array_t<ssize_t> &input) {\n auto buf = input.request();\n char *ptr = (char *) buf.ptr;\n ssize_t stride0 = input.strides(0), stride1 = input.strides(1);
\n\nfor (uint i = 0; i < state.shape[0]; i++) {\n for (uint j = 0; j < state.shape[1]; j++) {\n state(i, j) = (ssize_t &) ptr[i * stride0 + j * stride1];\n }\n}\n}
\n\n// Here we create a np.array in C++ and to return it into Python. Works like a charm!\npy::array_t<ssize_t> convert_result(const AutomatonState &state) {\n py::array_t<ssize_t> output(state.color.size());\n auto buf = output.request();\n ssize_t *ptr = (ssize_t *) buf.ptr;
\n\nfor (uint i = 0; i < state.shape[0]; i++) {\n for (uint j = 0; j < state.shape[1]; j++) {\n ptr[i * state.shape[1] + j] = state(i, j);\n }\n}\n\noutput.resize({state.shape[0], state.shape[1]});\n// printf(\"output shape: %d x %d\\n\", (int) output.shape(0), (int) output.shape(1));\nreturn output;\n}\n```
", "rawMarkdown": "An example of pybind11 interface usage:\n```\n// Parameters: a custom class and a list of List[Dict], List[Dict], Dict and Dict.\nvoid convert_all_rules(AutomatonParams ¶ms, const py::list ¶ms_list) {\n assert(params_list.size() == 4);\n \n for (const auto &item : params_list[0].cast<py::list>()) {\n params.global_rules.push_back(convert_rule(item.cast<py::dict>()));\n }\n \n for (const auto &item : params_list[1].cast<py::list>()) {\n params.ca_rules.push_back(convert_rule(item.cast<py::dict>()));\n }\n\n // We can even debug print whole objects using their __str__ method!\n // puts(std::string(params_list[2].cast<py::str>()).c_str());\n\n params.split_rule = convert_rule(params_list[2].cast<py::dict>());\n params.merge_rule = convert_rule(params_list[3].cast<py::dict>());\n}\n \n// This receives np.array and converts it into a custom format\nvoid convert_automaton_state(AutomatonState &state, const py::array_t<ssize_t> &input) {\n auto buf = input.request();\n char *ptr = (char *) buf.ptr;\n ssize_t stride0 = input.strides(0), stride1 = input.strides(1);\n \n for (uint i = 0; i < state.shape[0]; i++) {\n for (uint j = 0; j < state.shape[1]; j++) {\n state(i, j) = (ssize_t &) ptr[i * stride0 + j * stride1];\n }\n }\n}\n \n// Here we create a np.array in C++ and to return it into Python. Works like a charm!\npy::array_t<ssize_t> convert_result(const AutomatonState &state) {\n py::array_t<ssize_t> output(state.color.size());\n auto buf = output.request();\n ssize_t *ptr = (ssize_t *) buf.ptr;\n \n for (uint i = 0; i < state.shape[0]; i++) {\n for (uint j = 0; j < state.shape[1]; j++) {\n ptr[i * state.shape[1] + j] = state(i, j);\n }\n }\n \n output.resize({state.shape[0], state.shape[1]});\n // printf(\"output shape: %d x %d\\n\", (int) output.shape(0), (int) output.shape(1));\n return output;\n}\n```", "votes": 2 }, { "id": 865328, "postDate": "2020-05-28T14:57:16.557Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 864722, "postDate": "2020-05-28T06:45:48.493Z", "content": "A couple of words about my part of the solution mentioned in the post.\nI tried to solve puzzles, when a solution is the part (crop) of the puzzle itself. For each task I check, whether solutions are crops or not. If it is, I train a simple ML model on all possible crops. \nIt gave me 0.97 on the leaderboard, and 0.95 after ensembling with two public kernels.
\n\nDuring the last week of the competition, after teaming up with these great guys, I added the ability not only to cut crops from the input image, but also change colors.\nSo my part of the solution could possibly solve tasks, where the answer is a combination of cutting crop and color repainting, or just color repainting (in this case crop is equal to image itself).\nIt gave me 6 more tasks on the validation set, but unfortunately it didn't improve our public score.
\n\nI would also like to thank my team @alexfritz @artyomp @felipefonte99 ! You did a great job!
", "rawMarkdown": "A couple of words about my part of the solution mentioned in the post.\nI tried to solve puzzles, when a solution is the part (crop) of the puzzle itself. For each task I check, whether solutions are crops or not. If it is, I train a simple ML model on all possible crops. \nIt gave me 0.97 on the leaderboard, and 0.95 after ensembling with two public kernels.\n\nDuring the last week of the competition, after teaming up with these great guys, I added the ability not only to cut crops from the input image, but also change colors.\nSo my part of the solution could possibly solve tasks, where the answer is a combination of cutting crop and color repainting, or just color repainting (in this case crop is equal to image itself).\nIt gave me 6 more tasks on the validation set, but unfortunately it didn't improve our public score.\n\nI would also like to thank my team @alexfritz @artyomp @felipefonte99 ! You did a great job! \n", "votes": 3 }, { "id": 864319, "postDate": "2020-05-28T00:19:17.957Z", "content": "Here is my part mentioned above:
\n\nDescription\nMy DSL is based on 'operations'. It applies a lot of operations on the input and stores the result (prediction) of each operation. For example: crop the biggest object, crop the object of the less frequent color, flip some object, change background color, etc. etc. And also those operations combined. So it tries thousands of operations.
\n\nLoss\nWith thousands of predictions, which one should I choose? It's simple: for each prediction I calculate a loss based on how many parts of the training set of that task it solved. For example, if a loss for a specific prediction is 0, it means that operation solved all parts of the training set, so it must be an excellent prediction for the test part. So in the end I just select the 3 predictions with the smallest loss.
\n\nResult\nThis algorithm solved 5 tasks of the test set (score 0.95), all of them were tasks where the output size is smaller than the input. I also ensembled with two public kernels to solve 2 more tasks and get 0.93 on the leaderboard before teaming up with my great teammates.
", "rawMarkdown": "Here is my part mentioned above:\n\n**Description**\nMy DSL is based on 'operations'. It applies a lot of operations on the input and stores the result (prediction) of each operation. For example: crop the biggest object, crop the object of the less frequent color, flip some object, change background color, etc. etc. And also those operations combined. So it tries thousands of operations.\n\n**Loss**\nWith thousands of predictions, which one should I choose? It's simple: for each prediction I calculate a loss based on how many parts of the training set of that task it solved. For example, if a loss for a specific prediction is 0, it means that operation solved all parts of the training set, so it must be an excellent prediction for the test part. So in the end I just select the 3 predictions with the smallest loss.\n\n**Result**\nThis algorithm solved 5 tasks of the test set (score 0.95), all of them were tasks where the output size is smaller than the input. I also ensembled with two public kernels to solve 2 more tasks and get 0.93 on the leaderboard before teaming up with my great teammates.", "votes": 4, "replies": [ { "id": 864426, "postDate": "2020-05-28T02:02:39.733Z", "content": "Thanks for sharing. The Loss part is super interesting. wow.
", "rawMarkdown": "Thanks for sharing. The Loss part is super interesting. wow.", "votes": 1 } ] }, { "id": 864322, "postDate": "2020-05-28T00:19:36.913Z", "content": "That's very interesting. In your DSL language how does it cope with this like \"change all the shapes with the highest number of holes in them to green\" ?
\n\nThe way I tried to do it was look at the inputs and outputs, and try to find out what were the changes and what properties of the shapes that changed and then try to work out a rule.
\n\nAs well as trying lots of different brute force things too like symmetries etc.
", "rawMarkdown": "That's very interesting. In your DSL language how does it cope with this like \"change all the shapes with the highest number of holes in them to green\" ?\n\nThe way I tried to do it was look at the inputs and outputs, and try to find out what were the changes and what properties of the shapes that changed and then try to work out a rule. \n\nAs well as trying lots of different brute force things too like symmetries etc.", "replies": [ { "id": 865184, "postDate": "2020-05-28T13:02:25.913Z", "content": "I don't think there are tasks in the test set like this. My XGBoost approach could count holes by counting the number of connected components. This solved many tasks on train and validation, but didn't improve the score.
\n\nWe have to think in a more general way.
", "rawMarkdown": "I don't think there are tasks in the test set like this. My XGBoost approach could count holes by counting the number of connected components. This solved many tasks on train and validation, but didn't improve the score.\n\nWe have to think in a more general way." } ] }, { "id": 866635, "postDate": "2020-05-29T15:02:02.150Z", "rawMarkdown": "", "isDeleted": true }, { "id": 872107, "postDate": "2020-06-02T23:09:58.560Z", "content": "Conglats! thanks for sharing.
", "rawMarkdown": "Conglats! thanks for sharing.", "votes": 1 } ], "complete": true, "source": "live", "fetched_at": "2026-10-19T22:05:21Z", "competition_id": 18329, "competition_slug": "abstraction-and-reasoning-challenge", "team_id": 4421855, "solution_url": "/competitions/abstraction-and-reasoning-challenge/writeups/puzzlemaster-our-10th-place-solution-with-code" }