Thanks to everyone, organizers and fellow competitors, for an amazing competition. This is the first time I have actually dared to actually enter a Kaggle competition, despite the fact that I'm supposed to be good at machine learning and all that :-).
\n\nWe'll be posting a writeup soon, but I first want to thank my amazing teammates, @jerrybai1995, @devinwillmott, @jpmailoa, @mordechai1, who of course did all the actual work.
\n\nWe'll be posting a writeup soon, of course, but the only \"secret\" sauce I'll mention is:\n1. A ton of luck ... we weren't holding any methods back or anything like that, our performance each day by the last week was the best we could do at that point, and our models literally all finished training today.\n2. Having access to the compute infrastructure that being a CMU prof and Bosch Chief Scientist affords you and all that ... I think some of the other teams likely did much better on less. (The funny part, of course, is that my students constantly complain about not having enough compute...)\n3. Expect Graph Transformers to soon oust generic GNNs as the dominant deep learning approach to graph modeling :-).
\n\nThanks especially to the Quantum Uncertainty team, who were constant goalposts of excellence for us.
\n\nThanks everyone,\nZico Kolter
", "messages": [ { "id": 610737, "postDate": "2019-08-29T00:45:42.727Z", "content": "Thanks to everyone, organizers and fellow competitors, for an amazing competition. This is the first time I have actually dared to actually enter a Kaggle competition, despite the fact that I'm supposed to be good at machine learning and all that :-).
\n\nWe'll be posting a writeup soon, but I first want to thank my amazing teammates, @jerrybai1995, @devinwillmott, @jpmailoa, @mordechai1, who of course did all the actual work.
\n\nWe'll be posting a writeup soon, of course, but the only \"secret\" sauce I'll mention is:\n1. A ton of luck ... we weren't holding any methods back or anything like that, our performance each day by the last week was the best we could do at that point, and our models literally all finished training today.\n2. Having access to the compute infrastructure that being a CMU prof and Bosch Chief Scientist affords you and all that ... I think some of the other teams likely did much better on less. (The funny part, of course, is that my students constantly complain about not having enough compute...)\n3. Expect Graph Transformers to soon oust generic GNNs as the dominant deep learning approach to graph modeling :-).
\n\nThanks especially to the Quantum Uncertainty team, who were constant goalposts of excellence for us.
\n\nThanks everyone,\nZico Kolter
", "rawMarkdown": "Thanks to everyone, organizers and fellow competitors, for an amazing competition. This is the first time I have actually dared to actually enter a Kaggle competition, despite the fact that I'm supposed to be good at machine learning and all that :-).\n\nWe'll be posting a writeup soon, but I first want to thank my amazing teammates, @jerrybai1995, @devinwillmott, @jpmailoa, @mordechai1, who of course did all the actual work.\n\nWe'll be posting a writeup soon, of course, but the only \"secret\" sauce I'll mention is:\n1. A ton of luck ... we weren't holding any methods back or anything like that, our performance each day by the last week was the best we could do at that point, and our models literally all finished training today.\n2. Having access to the compute infrastructure that being a CMU prof and Bosch Chief Scientist affords you and all that ... I think some of the other teams likely did much better on less. (The funny part, of course, is that my students constantly complain about not having enough compute...)\n3. Expect Graph Transformers to soon oust generic GNNs as the dominant deep learning approach to graph modeling :-).\n\nThanks especially to the Quantum Uncertainty team, who were constant goalposts of excellence for us.\n\nThanks everyone,\nZico Kolter", "votes": 22 }, { "id": 610934, "postDate": "2019-08-29T03:08:34.907Z", "content": "Congratulations! We though about using papers with transformers, but discarded the idea.
", "rawMarkdown": "Congratulations! We though about using papers with transformers, but discarded the idea." }, { "id": 610797, "postDate": "2019-08-29T01:24:44.123Z", "content": "Congrats, professor! Very impressive work!
", "rawMarkdown": "Congrats, professor! Very impressive work! " }, { "id": 610775, "postDate": "2019-08-29T01:05:29.787Z", "content": "Congratulations! Looking forward to seeing how the sausage was made :-).
", "rawMarkdown": "Congratulations! Looking forward to seeing how the sausage was made :-). " }, { "id": 610741, "postDate": "2019-08-29T00:48:19.120Z", "content": "Congratulations on a terrific performance. Really looking forward to the writeup!
", "rawMarkdown": "Congratulations on a terrific performance. Really looking forward to the writeup!" }, { "id": 610743, "postDate": "2019-08-29T00:48:45.323Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 610934, "author_name": "Andrey Lukyanenko", "author_url": "", "post_date": "2019-08-29T03:08:34.907000", "content": "Congratulations! We though about using papers with transformers, but discarded the idea.
", "votes": 0, "replies": [] }, { "id": 610797, "author_name": "Youhan Lee", "author_url": "", "post_date": "2019-08-29T01:24:44.123000", "content": "Congrats, professor! Very impressive work!
", "votes": 0, "replies": [] }, { "id": 610775, "author_name": "Erik Thiede", "author_url": "", "post_date": "2019-08-29T01:05:29.787000", "content": "Congratulations! Looking forward to seeing how the sausage was made :-).
", "votes": 0, "replies": [] }, { "id": 610741, "author_name": "Joshua Colmer", "author_url": "", "post_date": "2019-08-29T00:48:19.120000", "content": "Congratulations on a terrific performance. Really looking forward to the writeup!
", "votes": 0, "replies": [] }, { "id": 610743, "author_name": "", "author_url": "", "post_date": "2019-08-29T00:48:45.323000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "610737": "Thanks to everyone, organizers and fellow competitors, for an amazing competition. This is the first time I have actually dared to actually enter a Kaggle competition, despite the fact that I'm supposed to be good at machine learning and all that :-).\n\nWe'll be posting a writeup soon, but I first want to thank my amazing teammates, @jerrybai1995, @devinwillmott, @jpmailoa, @mordechai1, who of course did all the actual work.\n\nWe'll be posting a writeup soon, of course, but the only \"secret\" sauce I'll mention is:\n1. A ton of luck ... we weren't holding any methods back or anything like that, our performance each day by the last week was the best we could do at that point, and our models literally all finished training today.\n2. Having access to the compute infrastructure that being a CMU prof and Bosch Chief Scientist affords you and all that ... I think some of the other teams likely did much better on less. (The funny part, of course, is that my students constantly complain about not having enough compute...)\n3. Expect Graph Transformers to soon oust generic GNNs as the dominant deep learning approach to graph modeling :-).\n\nThanks especially to the Quantum Uncertainty team, who were constant goalposts of excellence for us.\n\nThanks everyone,\nZico Kolter", "610934": "Congratulations! We though about using papers with transformers, but discarded the idea.", "610797": "Congrats, professor! Very impressive work! ", "610775": "Congratulations! Looking forward to seeing how the sausage was made :-). ", "610741": "Congratulations on a terrific performance. Really looking forward to the writeup!", "610743": "" } }