![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3067580%2Fbafc40a8cfb94c68761d97ec8531e61d%2Fkaggle_weights.png?generation=1567700298615562&alt=media\")
There's been several requests for some additional analysis on how to top models performed and I've tried to address some of these below. Let me know if there's some other simple things that you'd like to see.
\n\nA linear combination of the top 50 models, under the constraint that the weights should be positive and sum to 1, fitted to 50% of the test set, showed that only 7 models contributed with more than 1%, with the weights shown below.
\n\n
The top 6 models were from 'hybrid', '🤖 Quantum Uncertainty 🤖', '[ka.kr] Solve chem. together', 'Hyperspatial Engineers', 'DL guys' and 'Robin N' respectively, however the 12th placed submission by 'Team Bird Brain' contributed a lot to the fitted ensemble as well. The performance of the ensemble fitted on all coupling types ('Ensemble'), the ensemble fitted on each coupling type seperately ('Ensemble*') and the submitted solutions (individual contributions and the total score per coupling type) on the remaining 50% of the test set is shown in the table below:
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3067580%2F925ede7d4e16b38f4404507d7789829d%2Fkaggle_ensemble2.png?generation=1568043879263313&alt=media\")
When fitting weights to each coupling time separately, several other methods contributed to at least one of the ensembles:
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3067580%2F28e373e40d026325917fe1233484095e%2Fkaggle_ind_weights.png?generation=1568046129496263&alt=media\")
The figure below shows the correlation matrix (weighted by the mae per type) between these models. It's clear that the 'Team Bird Brain' model is relatively uncorrelated to the other models, which might explain their high weight in the ensemble.
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3067580%2Fcc9efe40fa8b955ad1eb4ac68b4204b9%2Fcorr.png?generation=1567703414827339&alt=media\")
There's been several requests for some additional analysis on how to top models performed and I've tried to address some of these below. Let me know if there's some other simple things that you'd like to see.
\n\nA linear combination of the top 50 models, under the constraint that the weights should be positive and sum to 1, fitted to 50% of the test set, showed that only 7 models contributed with more than 1%, with the weights shown below.
\n\n
The top 6 models were from 'hybrid', '🤖 Quantum Uncertainty 🤖', '[ka.kr] Solve chem. together', 'Hyperspatial Engineers', 'DL guys' and 'Robin N' respectively, however the 12th placed submission by 'Team Bird Brain' contributed a lot to the fitted ensemble as well. The performance of the ensemble fitted on all coupling types ('Ensemble'), the ensemble fitted on each coupling type seperately ('Ensemble*') and the submitted solutions (individual contributions and the total score per coupling type) on the remaining 50% of the test set is shown in the table below:
\n\n
When fitting weights to each coupling time separately, several other methods contributed to at least one of the ensembles:
\n\n
The figure below shows the correlation matrix (weighted by the mae per type) between these models. It's clear that the 'Team Bird Brain' model is relatively uncorrelated to the other models, which might explain their high weight in the ensemble.
\n\n
Awesome !!
", "rawMarkdown": "Awesome !!", "votes": null }, { "id": "619054", "postDate": "09/05/2019 19:36:16", "content": "Thank you very much, this is very interesting!
", "rawMarkdown": "Thank you very much, this is very interesting!", "votes": null }, { "id": "619107", "postDate": "09/05/2019 20:45:46", "content": "Great thanks! Exactly what I was looking for! (Being 55th and not in the final blend is disappointing though! ;) )
", "rawMarkdown": "Great thanks! Exactly what I was looking for! (Being 55th and not in the final blend is disappointing though! ;) )", "votes": null }, { "id": "619784", "postDate": "09/06/2019 15:12:03", "content": "Thank you for fulfilling our requests. This was a fantastic competition!
", "rawMarkdown": "Thank you for fulfilling our requests. This was a fantastic competition!", "votes": null }, { "id": "620345", "postDate": "09/07/2019 11:08:10", "content": "Very interesting thanks :)
", "rawMarkdown": "Very interesting thanks :)", "votes": null }, { "id": "622252", "postDate": "09/09/2019 12:32:58", "content": "It would be interesting to see how far you can go by computing weights type by type instead of globally for all types.
", "rawMarkdown": "It would be interesting to see how far you can go by computing weights type by type instead of globally for all types.", "votes": null }, { "id": "622258", "postDate": "09/09/2019 12:48:26", "content": "I can try and have a look, but might be a few days.
", "rawMarkdown": "I can try and have a look, but might be a few days.", "votes": null }, { "id": "622290", "postDate": "09/09/2019 13:16:15", "content": "Thanks, we are not in a hurry anymore...
", "rawMarkdown": "Thanks, we are not in a hurry anymore...", "votes": null }, { "id": "622418", "postDate": "09/09/2019 16:23:01", "content": "I added this to the post.
", "rawMarkdown": "I added this to the post.", "votes": null }, { "id": "622431", "postDate": "09/09/2019 16:41:45", "content": "Thanks, but I don't see the total. It should be better than when weights are the same across types.
", "rawMarkdown": "Thanks, but I don't see the total. It should be better than when weights are the same across types.", "votes": null }, { "id": "622453", "postDate": "09/09/2019 17:16:14", "content": "I guess its Ensemble*
", "rawMarkdown": "I guess its Ensemble*", "votes": null }, { "id": "622760", "postDate": "09/10/2019 03:53:42", "content": "Second table was updated. Tuning weights for each types achieved -3.453 ! Amazing !
", "rawMarkdown": "Second table was updated. Tuning weights for each types achieved -3.453 ! Amazing !", "votes": null }, { "id": "622864", "postDate": "09/10/2019 07:13:22", "content": "Fair enough!
", "rawMarkdown": "Fair enough!", "votes": null } ], "comments": [ { "id": 618961, "author_name": "mks2192", "author_url": "", "post_date": "09/05/2019 17:28:41", "content": "Awesome !!
", "votes": null, "replies": [] }, { "id": 619054, "author_name": "jpmailoa", "author_url": "", "post_date": "09/05/2019 19:36:16", "content": "Thank you very much, this is very interesting!
", "votes": null, "replies": [] }, { "id": 619107, "author_name": "optimo", "author_url": "", "post_date": "09/05/2019 20:45:46", "content": "Great thanks! Exactly what I was looking for! (Being 55th and not in the final blend is disappointing though! ;) )
", "votes": null, "replies": [] }, { "id": 619784, "author_name": "bigironsphere", "author_url": "", "post_date": "09/06/2019 15:12:03", "content": "Thank you for fulfilling our requests. This was a fantastic competition!
", "votes": null, "replies": [] }, { "id": 620345, "author_name": "zikazika", "author_url": "", "post_date": "09/07/2019 11:08:10", "content": "Very interesting thanks :)
", "votes": null, "replies": [] }, { "id": 622252, "author_name": "cpmpml", "author_url": "", "post_date": "09/09/2019 12:32:58", "content": "It would be interesting to see how far you can go by computing weights type by type instead of globally for all types.
", "votes": null, "replies": [ { "id": 622258, "author_name": "larsbratholm", "author_url": "", "post_date": "09/09/2019 12:48:26", "content": "I can try and have a look, but might be a few days.
", "votes": null, "replies": [] }, { "id": 622290, "author_name": "cpmpml", "author_url": "", "post_date": "09/09/2019 13:16:15", "content": "Thanks, we are not in a hurry anymore...
", "votes": null, "replies": [] }, { "id": 622418, "author_name": "larsbratholm", "author_url": "", "post_date": "09/09/2019 16:23:01", "content": "I added this to the post.
", "votes": null, "replies": [] }, { "id": 622431, "author_name": "cpmpml", "author_url": "", "post_date": "09/09/2019 16:41:45", "content": "Thanks, but I don't see the total. It should be better than when weights are the same across types.
", "votes": null, "replies": [] }, { "id": 622453, "author_name": "mks2192", "author_url": "", "post_date": "09/09/2019 17:16:14", "content": "I guess its Ensemble*
", "votes": null, "replies": [] }, { "id": 622760, "author_name": "toshik", "author_url": "", "post_date": "09/10/2019 03:53:42", "content": "Second table was updated. Tuning weights for each types achieved -3.453 ! Amazing !
", "votes": null, "replies": [] }, { "id": 622864, "author_name": "cpmpml", "author_url": "", "post_date": "09/10/2019 07:13:22", "content": "Fair enough!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "618947": "There's been several requests for some additional analysis on how to top models performed and I've tried to address some of these below. Let me know if there's some other simple things that you'd like to see.\n\nA linear combination of the top 50 models, under the constraint that the weights should be positive and sum to 1, fitted to 50% of the test set, showed that only 7 models contributed with more than 1%, with the weights shown below.\n\n\n\nThe top 6 models were from 'hybrid', '🤖 Quantum Uncertainty 🤖', '[ka.kr] Solve chem. together', 'Hyperspatial Engineers', 'DL guys' and 'Robin N' respectively, however the 12th placed submission by 'Team Bird Brain' contributed a lot to the fitted ensemble as well. The performance of the ensemble fitted on all coupling types ('Ensemble'), the ensemble fitted on each coupling type seperately ('Ensemble*') and the submitted solutions (individual contributions and the total score per coupling type) on the remaining 50% of the test set is shown in the table below:\n\n\n\nWhen fitting weights to each coupling time separately, several other methods contributed to at least one of the ensembles:\n\n\n\n\nThe figure below shows the correlation matrix (weighted by the mae per type) between these models. It's clear that the 'Team Bird Brain' model is relatively uncorrelated to the other models, which might explain their high weight in the ensemble.\n\n", "618961": "Awesome !!", "619054": "Thank you very much, this is very interesting!", "619107": "Great thanks! Exactly what I was looking for! (Being 55th and not in the final blend is disappointing though! ;) )", "619784": "Thank you for fulfilling our requests. This was a fantastic competition!", "620345": "Very interesting thanks :)", "622252": "It would be interesting to see how far you can go by computing weights type by type instead of globally for all types.", "622258": "I can try and have a look, but might be a few days.", "622290": "Thanks, we are not in a hurry anymore...", "622418": "I added this to the post.", "622431": "Thanks, but I don't see the total. It should be better than when weights are the same across types.", "622453": "I guess its Ensemble*", "622760": "Second table was updated. Tuning weights for each types achieved -3.453 ! Amazing !", "622864": "Fair enough!" }, "source": "meta" }