{
  "id": 100257,
  "title": "Is there a faster way to implement Giba's features in Python？",
  "url": "/competitions/champs-scalar-coupling/discussion/100257",
  "author_name": "",
  "post_date": "2019-07-17T12:49:39.358289800Z",
  "votes": 1,
  "comment_count": 2,
  "views": 0,
  "content": "<p>```\ntrain_test[, pos := 1:.N, by=c(\"molecule_name\",\"atom_index_0\") ]</p>\n\n<p>sort_dist &lt;- function( par, typ, k=\"H\" ){\n  res &lt;- matrix( 999, nrow=length(par), ncol=13 )\n  par[ typ!=k ] &lt;- 999\n  for( i in 1:nrow(res) ){\n    res[i, 1:length(par) ] &lt;- par\n    res[i, i ] &lt;- 999\n    res[i,] &lt;- res[i, order(res[i,],decreasing=F) ]\n  }\n  res[res==999] &lt;- NA\n  return( list( res[,1],res[,2],res[,3],res[,4],res[,5] ) )\n}</p>\n\n<p>dt &lt;- train_test[, sort_dist(par=dist_xyz,typ=structure_atom_1,k=\"H\") ,\n                 keyby=c(\"molecule_name\",\"atom_index_0\") ]\ndt[, pos := 1:.N, by=c(\"molecule_name\",\"atom_index_0\") ]\nsetkeyv( dt, c(\"molecule_name\",\"atom_index_0\",\"pos\")  )</p>\n\n<p>train_test[, adH1 := dt[J(train_test$molecule_name, train_test$atom_index_0, train_test$pos)]$V1 ]\ntrain_test[, adH2 := dt[J(train_test$molecule_name, train_test$atom_index_0, train_test$pos)]$V2 ]\ntrain_test[, adH3 := dt[J(train_test$molecule_name, train_test$atom_index_0, train_test$pos)]$V3 ]\ntrain_test[, adH4 := dt[J(train_test$molecule_name, train_test$atom_index_0, train_test$pos)]$V4 ]\nhead(train_test)\n```</p>\n\n<p>Especially these code which calculate the top distance groupby(\"molecule_name\",\"atom_index_0\"). </p>\n\n<p>Notice that before order:\n<code>\nres[i, i ] &amp;lt;- 999 ###when cal the topks of ith item's in group , its dist_xyz will not participate in \n                          ###sorting. <br>\n</code></p>\n\n<p>I'm implementing this by pandas.df.apply(), but it was tooooooooo slow……Anybody has a better idea？</p>",
  "messages": [
    {
      "id": "578183",
      "postDate": "07/17/2019 12:49:39",
      "content": "<p>```\ntrain_test[, pos := 1:.N, by=c(\"molecule_name\",\"atom_index_0\") ]</p>\n\n<p>sort_dist &lt;- function( par, typ, k=\"H\" ){\n  res &lt;- matrix( 999, nrow=length(par), ncol=13 )\n  par[ typ!=k ] &lt;- 999\n  for( i in 1:nrow(res) ){\n    res[i, 1:length(par) ] &lt;- par\n    res[i, i ] &lt;- 999\n    res[i,] &lt;- res[i, order(res[i,],decreasing=F) ]\n  }\n  res[res==999] &lt;- NA\n  return( list( res[,1],res[,2],res[,3],res[,4],res[,5] ) )\n}</p>\n\n<p>dt &lt;- train_test[, sort_dist(par=dist_xyz,typ=structure_atom_1,k=\"H\") ,\n                 keyby=c(\"molecule_name\",\"atom_index_0\") ]\ndt[, pos := 1:.N, by=c(\"molecule_name\",\"atom_index_0\") ]\nsetkeyv( dt, c(\"molecule_name\",\"atom_index_0\",\"pos\")  )</p>\n\n<p>train_test[, adH1 := dt[J(train_test$molecule_name, train_test$atom_index_0, train_test$pos)]$V1 ]\ntrain_test[, adH2 := dt[J(train_test$molecule_name, train_test$atom_index_0, train_test$pos)]$V2 ]\ntrain_test[, adH3 := dt[J(train_test$molecule_name, train_test$atom_index_0, train_test$pos)]$V3 ]\ntrain_test[, adH4 := dt[J(train_test$molecule_name, train_test$atom_index_0, train_test$pos)]$V4 ]\nhead(train_test)\n```</p>\n\n<p>Especially these code which calculate the top distance groupby(\"molecule_name\",\"atom_index_0\"). </p>\n\n<p>Notice that before order:\n<code>\nres[i, i ] &amp;lt;- 999 ###when cal the topks of ith item's in group , its dist_xyz will not participate in \n                          ###sorting. <br>\n</code></p>\n\n<p>I'm implementing this by pandas.df.apply(), but it was tooooooooo slow……Anybody has a better idea？</p>",
      "rawMarkdown": "```\ntrain_test[, pos := 1:.N, by=c(\"molecule_name\",\"atom_index_0\") ]\n\nsort_dist &lt;- function( par, typ, k=\"H\" ){\n  res &lt;- matrix( 999, nrow=length(par), ncol=13 )\n  par[ typ!=k ] &lt;- 999\n  for( i in 1:nrow(res) ){\n    res[i, 1:length(par) ] &lt;- par\n    res[i, i ] &lt;- 999\n    res[i,] &lt;- res[i, order(res[i,],decreasing=F) ]\n  }\n  res[res==999] &lt;- NA\n  return( list( res[,1],res[,2],res[,3],res[,4],res[,5] ) )\n}\n\ndt &lt;- train_test[, sort_dist(par=dist_xyz,typ=structure_atom_1,k=\"H\") ,\n                 keyby=c(\"molecule_name\",\"atom_index_0\") ]\ndt[, pos := 1:.N, by=c(\"molecule_name\",\"atom_index_0\") ]\nsetkeyv( dt, c(\"molecule_name\",\"atom_index_0\",\"pos\")  )\n\ntrain_test[, adH1 := dt[J(train_test$molecule_name, train_test$atom_index_0, train_test$pos)]$V1 ]\ntrain_test[, adH2 := dt[J(train_test$molecule_name, train_test$atom_index_0, train_test$pos)]$V2 ]\ntrain_test[, adH3 := dt[J(train_test$molecule_name, train_test$atom_index_0, train_test$pos)]$V3 ]\ntrain_test[, adH4 := dt[J(train_test$molecule_name, train_test$atom_index_0, train_test$pos)]$V4 ]\nhead(train_test)\n```\n\n\nEspecially these code which calculate the top distance groupby(\"molecule_name\",\"atom_index_0\"). \n\nNotice that before order:\n```\nres[i, i ] &lt;- 999 ###when cal the topks of ith item's in group , its dist_xyz will not participate in \n                          ###sorting.  \n```\n\nI'm implementing this by pandas.df.apply(), but it was tooooooooo slow……Anybody has a better idea？",
      "votes": null
    },
    {
      "id": "578218",
      "postDate": "07/17/2019 13:37:07",
      "content": "<p>Check this: <a href=\"https://www.kaggle.com/saurabh7/compact-feature-engineering-lgbm-py-version#Distances-to-Nearest-neighbors\">https://www.kaggle.com/saurabh7/compact-feature-engineering-lgbm-py-version#Distances-to-Nearest-neighbors</a>\nI am using <code>rank</code> to rank by atoms. Haven't timed it, but doesn't take more the 5 mins.</p>",
      "rawMarkdown": "Check this: https://www.kaggle.com/saurabh7/compact-feature-engineering-lgbm-py-version#Distances-to-Nearest-neighbors\nI am using `rank` to rank by atoms. Haven't timed it, but doesn't take more the 5 mins.",
      "votes": null
    },
    {
      "id": "578375",
      "postDate": "07/17/2019 16:41:25",
      "content": "<p>Thx a lot. Actually, i am fingding a way to implement \"res[i, i ] &lt;- 999\" when doing sorting .  But maybe there is no influnce to include the res[i,i]……</p>",
      "rawMarkdown": "Thx a lot. Actually, i am fingding a way to implement \"res[i, i ] &lt;- 999\" when doing sorting .  But maybe there is no influnce to include the res[i,i]……",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 578218,
      "author_name": "saurabh7",
      "author_url": "",
      "post_date": "07/17/2019 13:37:07",
      "content": "<p>Check this: <a href=\"https://www.kaggle.com/saurabh7/compact-feature-engineering-lgbm-py-version#Distances-to-Nearest-neighbors\">https://www.kaggle.com/saurabh7/compact-feature-engineering-lgbm-py-version#Distances-to-Nearest-neighbors</a>\nI am using <code>rank</code> to rank by atoms. Haven't timed it, but doesn't take more the 5 mins.</p>",
      "votes": null,
      "replies": [
        {
          "id": 578375,
          "author_name": "sparkingarthur",
          "author_url": "",
          "post_date": "07/17/2019 16:41:25",
          "content": "<p>Thx a lot. Actually, i am fingding a way to implement \"res[i, i ] &lt;- 999\" when doing sorting .  But maybe there is no influnce to include the res[i,i]……</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "578183": "```\ntrain_test[, pos := 1:.N, by=c(\"molecule_name\",\"atom_index_0\") ]\n\nsort_dist &lt;- function( par, typ, k=\"H\" ){\n  res &lt;- matrix( 999, nrow=length(par), ncol=13 )\n  par[ typ!=k ] &lt;- 999\n  for( i in 1:nrow(res) ){\n    res[i, 1:length(par) ] &lt;- par\n    res[i, i ] &lt;- 999\n    res[i,] &lt;- res[i, order(res[i,],decreasing=F) ]\n  }\n  res[res==999] &lt;- NA\n  return( list( res[,1],res[,2],res[,3],res[,4],res[,5] ) )\n}\n\ndt &lt;- train_test[, sort_dist(par=dist_xyz,typ=structure_atom_1,k=\"H\") ,\n                 keyby=c(\"molecule_name\",\"atom_index_0\") ]\ndt[, pos := 1:.N, by=c(\"molecule_name\",\"atom_index_0\") ]\nsetkeyv( dt, c(\"molecule_name\",\"atom_index_0\",\"pos\")  )\n\ntrain_test[, adH1 := dt[J(train_test$molecule_name, train_test$atom_index_0, train_test$pos)]$V1 ]\ntrain_test[, adH2 := dt[J(train_test$molecule_name, train_test$atom_index_0, train_test$pos)]$V2 ]\ntrain_test[, adH3 := dt[J(train_test$molecule_name, train_test$atom_index_0, train_test$pos)]$V3 ]\ntrain_test[, adH4 := dt[J(train_test$molecule_name, train_test$atom_index_0, train_test$pos)]$V4 ]\nhead(train_test)\n```\n\n\nEspecially these code which calculate the top distance groupby(\"molecule_name\",\"atom_index_0\"). \n\nNotice that before order:\n```\nres[i, i ] &lt;- 999 ###when cal the topks of ith item's in group , its dist_xyz will not participate in \n                          ###sorting.  \n```\n\nI'm implementing this by pandas.df.apply(), but it was tooooooooo slow……Anybody has a better idea？",
    "578218": "Check this: https://www.kaggle.com/saurabh7/compact-feature-engineering-lgbm-py-version#Distances-to-Nearest-neighbors\nI am using `rank` to rank by atoms. Haven't timed it, but doesn't take more the 5 mins.",
    "578375": "Thx a lot. Actually, i am fingding a way to implement \"res[i, i ] &lt;- 999\" when doing sorting .  But maybe there is no influnce to include the res[i,i]……"
  },
  "source": "meta"
}