{
  "id": 336124,
  "title": "13th place solution (GNN)",
  "content": "<p>First of all, this competition would have been better without the leak.<br>\nIt is very sad that because of the leak, we can no longer distinguish between a good solution and a overfitting solution.</p>\n<h1>Summary</h1>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6958652%2F8709eae7598a8c4aa5f131aff0f0d128%2Fsolution.jpg?generation=1657372988563552&amp;alt=media\" alt=\"\"></p>\n<p>My solution consists of four stages</p>\n<ul>\n<li>Candidate generation<ul>\n<li>1st stage: Create candidates</li>\n<li>2nd stage: Filtering by LightGBM</li></ul></li>\n<li>Matching<ul>\n<li>3rd stage: match prediction by xlm-roberta-base and mdeberta-base-v3</li></ul></li>\n<li>PostProcessing<ul>\n<li>4th stage: Node Classification by GNN</li></ul></li>\n</ul>\n<h1>Cross validation strategy</h1>\n<p>GroupKFold(n_splits=2, group='point_of_interest')<br>\nThe CV score of my solution is 0.920</p>\n<h1>1st stage</h1>\n<ul>\n<li><p>Generates 60 candidates per id</p>\n<ul>\n<li>30 candidate neighborhoods per id using cosine similarity of text embedding by Tfidf<ul>\n<li>text: name, categories, address, city, state in a one sentence</li></ul></li>\n<li>30 candidate neighborhoods by haversine distance for per id</li></ul></li>\n</ul>\n<h1>2nd stage</h1>\n<ul>\n<li>Reduce to 3.7M pairs with high match probability using LightGBM</li>\n<li>Features<ul>\n<li>Missing value or not(name, cat)</li>\n<li>jaro(name, categories, address, city, state, zip, url, phone)</li>\n<li>leven(name, categories, address, city, state, zip, url, phone)</li>\n<li>haversine</li>\n<li>cos sim(tfidf embedding)</li>\n<li>mean top_k haversine, tfidf (k=5, 10, 15)</li></ul></li>\n<li>Using ForestInference</li>\n<li>Max IoU: 0.971</li>\n</ul>\n<h1>3rd stage</h1>\n<ul>\n<li>Predicts matche using a language model like DITTO</li>\n<li>model: xlm-roberta-base and mdeberta-base-v3</li>\n<li>Input<ul>\n<li>text: name_1 + [SEP] + name_2 + [COL] + categories_1 + [SEP] + categories_2 + [COL] + address_1 + [SEP] + address_2 + [COL] + city_1 + [SEP] + city_2 + [COL] + state_1 + [SEP] + state_2 + [COL] + zip_1 + [SEP] + zip_2</li>\n<li>numerical_feature</li>\n<li>haversine</li>\n<li>mean top_k haversine(k=5, 10, 15)</li></ul></li>\n<li>Training method<ul>\n<li>multitask learning: match and country (+0.003)</li>\n<li>augmentation</li></ul></li>\n</ul>\n<pre><code>import nlpaug as naw\n    aug_list = [\n  naw.RandomWordAug(action='swap', aug_p=0.1), \n  naw.RandomWordAug(action='delete', aug_p=0.1), \n  naw.SplitAug(aug_p=0.1)\n]\n</code></pre>\n<ul>\n<li>flip id_1, id_2</li>\n<li>TTA: flip id_1, id_2</li>\n</ul>\n<h1>4th stage</h1>\n<p>This stage is the most important part of my solution.<br>\nBy using GNN, both CV and LB are improved by about 0.02x.  </p>\n<p>Predicts the matching id for id_1 using GNN.</p>\n<ol>\n<li>Create a 2-hop-subgraph for each id_1(max iou: 0.993)  <br>\n<strong>example</strong> , Suppose the following table is given and we predict the id matching A.</li>\n</ol>\n<table>\n<thead>\n<tr>\n<th>id_1</th>\n<th>id_2</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>A</td>\n<td>B</td>\n</tr>\n<tr>\n<td>B</td>\n<td>A</td>\n</tr>\n<tr>\n<td>B</td>\n<td>C</td>\n</tr>\n<tr>\n<td>B</td>\n<td>D</td>\n</tr>\n<tr>\n<td>C</td>\n<td>D</td>\n</tr>\n<tr>\n<td>D</td>\n<td>B</td>\n</tr>\n<tr>\n<td>E</td>\n<td>A</td>\n</tr>\n<tr>\n<td>E</td>\n<td>F</td>\n</tr>\n<tr>\n<td>F</td>\n<td>E</td>\n</tr>\n</tbody>\n</table>\n<p>In this case, the 2-hop-subgraph for A is as follows(Think in terms of undirected graphs)</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6958652%2F26622119da7e573aa087ba1e4e942ff6%2FScreenshot%20from%202022-07-09%2021-30-51.png?generation=1657371037868405&amp;alt=media\" alt=\"\"></p>\n<ol>\n<li>Predict whether each node matches id_1</li>\n<li>Nodes that exceed the threshold are set as matches for id_1</li>\n</ol>\n<h2>Details</h2>\n<ul>\n<li>Model<br>\nThis model is based on the top solution of the Japanese competition platform atmaCup.</li>\n</ul>\n<pre><code>class SimpleGCN(nn.Module):\n    def __init__(self,\n                 num_node_features: int,\n                 num_edge_features: int,\n                #  num_classes: int,\n                 deg):\n        super(SimpleGCN, self).__init__()\n\n        aggregators = ['mean', 'min', 'max', 'std']\n        scalers = ['identity', 'amplification', 'attenuation']\n\n        towers = 4\n        pre_layer_num = 2\n        post_layer_num = 2\n        divide_input=False\n        self.convs = ModuleList()\n        self.batch_norms = ModuleList()\n        conv = PNAConv(in_channels=num_node_features, out_channels=32,\n                                    aggregators=aggregators, scalers=scalers, deg=deg,\n                                    edge_dim=num_edge_features, towers=towers, pre_layers=pre_layer_num, post_layers=post_layer_num,\n                                    divide_input=divide_input)\n        self.batch_norms.append(BatchNorm(32))\n        self.convs.append(conv)\n\n        for j in [32, 64, 128]:\n            conv = PNAConv(in_channels=j, out_channels=j*2, aggregators=aggregators, scalers=scalers, deg=deg, edge_dim=num_edge_features, towers=towers, pre_layers=pre_layer_num, post_layers=post_layer_num, divide_input=divide_input)\n            self.convs.append(conv)\n            self.batch_norms.append(BatchNorm(j*2))\n\n        self.dropout = nn.Dropout(0.2)\n        self.linear = nn.Linear(j*2, 1)\n\n    def forward(self, data):\n        x = data.x\n        edge_index = data.edge_index\n        edge_attr = data.edge_attr\n\n        for conv, batch_norm in zip(self.convs, self.batch_norms):\n            x = F.relu(batch_norm(conv(x, edge_index, edge_attr)))\n\n        x = self.dropout(x)\n        x = self.linear(x).flatten()\n        return x\n</code></pre>\n<ul>\n<li>node features<ul>\n<li>marker(I named it): 1 if the node corresponds to an id_1, 0 if it is a candidate.  <br>\nThe presence of this feature allows us to solve for node classification rather than link prediction.</li>\n<li>Missing value or not(name, categories, address, city, state, zip, url, phone)</li></ul></li>\n<li>edge features<ul>\n<li>mdeberta pred</li>\n<li>xlm-roberta pred</li>\n<li>lgbm pred</li>\n<li>jaro (address, city, state, zip, url, phone, categories)</li></ul></li>\n<li>loss function<ul>\n<li>IouLoss for each graph and the binary cross-entropy of the entire batch are the loss functions</li></ul></li>\n</ul>\n<pre><code>def iou_loss(logits, data):\n    sizes = degree(data.batch, dtype=torch.long).tolist()\n\n    logit_list = logits.sigmoid().split(sizes)\n    target_list = data.y.split(sizes)\n\n    loss = 0\n    for y_pred, y_target, group_size in zip(logit_list, target_list, data.group_size):\n\n        intersection = (y_pred * y_target).sum()\n        total = (y_pred + y_target).sum() + (group_size - y_target.sum())\n        union = total - intersection\n        loss += 1 - (intersection / union)\n\n    return loss / data.num_graphs\n\nloss = iou_loss + 0.1*BCELoss\n</code></pre>\n<h2>Advantages of GNN</h2>\n<ul>\n<li>Creating a 2-hop-subgraph greatly improved maxiou.</li>\n<li>Iou loss made it possible to optimize the competition metric directly.</li>\n<li>Fast inference</li>\n</ul>\n<h1>Public Score Timeline</h1>\n<ol>\n<li>20 candidates per id and mdeberta and lgbm stacking: 0.907</li>\n<li>60 candidates per id and xlm-roberta-base+mdeberta and lgbm stacking : 0.924</li>\n<li>60 candidates per id and xlm-roberta-base + mdeberta and GNN Post Processing: 0.946</li>\n</ol>",
  "comments": [
    {
      "id": 1849980,
      "author_name": "Patrick Chan",
      "author_url": "",
      "post_date": "2022-07-10T01:25:29.133000",
      "content": "<p>Great share! Personally I like your approach more than the other solutions. </p>",
      "votes": 4,
      "replies": []
    },
    {
      "id": 1900809,
      "author_name": "Try Harder",
      "author_url": "",
      "post_date": "2022-08-16T09:12:55.970000",
      "content": "<p>Great share!  Could you share your code about train and inference? Thank you so much!</p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 1850361,
      "author_name": "KKY",
      "author_url": "",
      "post_date": "2022-07-10T10:54:19.257000",
      "content": "<p>Greeeeeeate Work!  One quesition,   in stage4,  on 2-hop-subgraph,  seems only has the edge features in direct linked edge,  how to deal with the others edge?    Do you consider to share the inference code? </p>",
      "votes": 2,
      "replies": [
        {
          "id": 1850374,
          "author_name": "213tubo",
          "author_url": "",
          "post_date": "2022-07-10T11:02:35.560000",
          "content": "<p>Thanks for the question.</p>\n<p>2-hop-subgraph is created by considering the graph as an undirected graph.<br>\nThe implementation uses pytorch_geometric and can be written as follows<br>\n<a href=\"https://pytorch-geometric.readthedocs.io/en/latest/modules/utils.html#torch_geometric.utils.k_hop_subgraph\" target=\"_blank\">https://pytorch-geometric.readthedocs.io/en/latest/modules/utils.html#torch_geometric.utils.k_hop_subgraph</a></p>\n<pre><code>from torch_geometric.utils import k_hop_subgraph\n\nnodes = k_hop_subgraph(node_id, 2, G.edge_index, flow='target_to_source')[0]\nsubgraph = G.subgraph(nodes)\n</code></pre>\n<p>Sorry if this didn't answer your question.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1850376,
          "author_name": "213tubo",
          "author_url": "",
          "post_date": "2022-07-10T11:05:10.483000",
          "content": "<p>I'm thinking of sharing the inference code, but I'd like to organize it a bit before I share it with you.<br>\nThe current code is very dirty.😂</p>",
          "votes": 4,
          "replies": []
        },
        {
          "id": 1852672,
          "author_name": "Bilzard",
          "author_url": "",
          "post_date": "2022-07-12T08:33:11.683000",
          "content": "<p><a href=\"https://www.kaggle.com/evilpsycho42\" target=\"_blank\">@evilpsycho42</a> </p>\n<blockquote>\n  <p>how to deal with the others edge?</p>\n</blockquote>\n<p>Since the author saids he/she deals with the target task as a node classification, I think we don't have to consider the relation between non-directly-connected nodes.</p>\n<blockquote>\n  <p>The presence of this feature allows us to solve for node classification rather than link prediction.</p>\n</blockquote>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1852681,
          "author_name": "Bilzard",
          "author_url": "",
          "post_date": "2022-07-12T08:45:54.383000",
          "content": "<p><a href=\"https://www.kaggle.com/tubotubo\" target=\"_blank\">@tubotubo</a> I'm also interested in how the whole implementation (especially on GNN part) is done.<br>\nI don't care what the code looks like, the code would be very helpful.</p>\n<p>Thanks for sharing your solution.<br>\nI can't believe this all is done by only one person!</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1852725,
          "author_name": "213tubo",
          "author_url": "",
          "post_date": "2022-07-12T09:54:17.587000",
          "content": "<p><a href=\"https://www.kaggle.com/tatamikenn\" target=\"_blank\">@tatamikenn</a> </p>\n<blockquote>\n  <p>how to deal with the others edge?</p>\n  <p>Since the author saids he/she deals with the target task as a node classification, I think we don't have to consider the relation between non-directly-connected nodes.</p>\n  <blockquote>\n    <p>The presence of this feature allows us to solve for node classification rather than link prediction.</p>\n  </blockquote>\n</blockquote>\n<p>Thank you for supplementing my explanation.</p>\n<blockquote>\n  <p>I'm also interested in how the whole implementation (especially on GNN part) is done.<br>\n  I don't care what the code looks like, the code would be very helpful.</p>\n  <p>Thanks for sharing your solution.<br>\n  I can't believe this all is done by only one person!</p>\n</blockquote>\n<p>Thank you for complimenting my solution!<br>\nI will release it to the public only after modifying the code.<br>\nHowever, since I am not sure when I will be able to release the code, I have shared it privately with you.</p>\n<p>I am glad that you are interested in my solution.</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 1850184,
      "author_name": "MarBH666",
      "author_url": "",
      "post_date": "2022-07-10T07:19:18.147000",
      "content": "<p>you did a good job👍</p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 1850037,
      "author_name": "Young for you",
      "author_url": "",
      "post_date": "2022-07-10T03:01:59.753000",
      "content": "<p>Wow Great solution.  I am only your little version</p>",
      "votes": 2,
      "replies": [
        {
          "id": 1850040,
          "author_name": "213tubo",
          "author_url": "",
          "post_date": "2022-07-10T03:07:13.133000",
          "content": "<p>Thank you. It is nice to be admired by a strong Grandmaster like you!</p>\n<p>I am very interested in your solution too.<br>\nDo you plan to share your solution?<br>\nCan you give me a brief explanation of your solution?</p>",
          "votes": 2,
          "replies": []
        },
        {
          "id": 1850043,
          "author_name": "Young for you",
          "author_url": "",
          "post_date": "2022-07-10T03:13:21.763000",
          "content": "<p>My solution only your stage 1+stage2+the last stage still Lgb😂. You did a great job. I feel regret for you because of the silly \"leak\"</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1850045,
          "author_name": "213tubo",
          "author_url": "",
          "post_date": "2022-07-10T03:16:22.483000",
          "content": "<p>Thank you for sharing your solution!<br>\nleak is very sad😭. I hope the second Foursquare competition will be held.</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 1858756,
      "author_name": "Bilal Suppal",
      "author_url": "",
      "post_date": "2022-07-17T07:22:29.573000",
      "content": "<p>The information provided is so clear and easy to understand, Awesome post :)<br>\nLooking forward to see more !!!<br>\nlet's grow together</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1851508,
      "author_name": "yo4p",
      "author_url": "",
      "post_date": "2022-07-11T10:02:43.703000",
      "content": "<p>I like your approach very much. Thank you for sharing!</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1850381,
      "author_name": "Ethan",
      "author_url": "",
      "post_date": "2022-07-10T11:10:52.553000",
      "content": "<p>Thanks for sharing. GNN is really impressive!</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1850340,
      "author_name": "KKY",
      "author_url": "",
      "post_date": "2022-07-10T10:41:09.957000",
      "content": "<p>Very impressive solution!</p>\n<p>thanks for share.</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1849487,
      "author_name": "DeepUnderstanding",
      "author_url": "",
      "post_date": "2022-07-09T15:13:02.020000",
      "content": "<p>Your approach with GNNs is really impressive!<br>\nCongratulations!</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 2592341,
      "author_name": "Rim Zakhama",
      "author_url": "",
      "post_date": "2024-01-08T13:56:39.017000",
      "content": "<p>Thank you for sharing. Can you explain your approach for handling missing values, please?</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2528194,
      "author_name": "H210164",
      "author_url": "",
      "post_date": "2023-11-17T07:15:20.463000",
      "content": "<p>Great share! I like your solution very much! :)</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2081007,
      "author_name": "213tubo",
      "author_url": "",
      "post_date": "2022-12-30T18:10:57.567000",
      "content": "<p>I have decided not to share the code because the data is no longer available.</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1861341,
      "author_name": "Y-Haneji",
      "author_url": "",
      "post_date": "2022-07-19T01:26:41.157000",
      "content": "<p><a href=\"https://www.kaggle.com/tubotubo\" target=\"_blank\">@tubotubo</a> <br>\nThank you for sharing a great solution, and I appreciate your new (meaning the version without leak) score and prize!<br>\nCould you give me some example of input objects to GNN or dataset implementation? Of course, the figure helped me a lot with understanding the idea, but I can't imagine how to convert the pair table to PyTorch dataset representing the graph…</p>",
      "votes": 0,
      "replies": [
        {
          "id": 1861366,
          "author_name": "213tubo",
          "author_url": "",
          "post_date": "2022-07-19T01:54:50.367000",
          "content": "<p><a href=\"https://www.kaggle.com/hanejiyuto\" target=\"_blank\">@hanejiyuto</a> <br>\nThank you for your question.<br>\nI have implemented GNN using <a href=\"https://pytorch-geometric.readthedocs.io/en/latest/index.html\" target=\"_blank\">pytorch_geometric</a> and <a href=\"https://networkx.org/\" target=\"_blank\">networkx</a>.</p>\n<p>Follow this procedure to create the inputs to the GNN.</p>\n<h3>1. Create a large graph in the following way.</h3>\n<p>df_pair</p>\n<table>\n<thead>\n<tr>\n<th>id_1</th>\n<th>id_2</th>\n<th>mdeberta</th>\n<th>xlm-roberta</th>\n<th>lgbm</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>A</td>\n<td>B</td>\n<td>0.9</td>\n<td>0.9</td>\n<td>0.4</td>\n</tr>\n<tr>\n<td>…</td>\n<td>…</td>\n<td>…</td>\n<td>…</td>\n<td>…</td>\n</tr>\n<tr>\n<td>Z</td>\n<td>A</td>\n<td>0.2</td>\n<td>0.5</td>\n<td>0.6</td>\n</tr>\n</tbody>\n</table>\n<pre><code>import networkx as nx\nfrom torch_geometric.utils.convert import from_networkx\n\ndef build_graph(df_pair, weight_att_cols):\n      nx_G = nx.from_pandas_edgelist(\n               df_pair, \n               source='id_1',\n               target='id_2',\n               edge_attr=weight_att_cols,\n               create_using=nx.DiGraph()\n      )\n\n      pyg_G = from_networkx(nx_G, all, all)\n\n      return pyg_G\n</code></pre>\n<h3>2. Partition a large graph into connected components</h3>\n<h3>3. Create a 2-hop-subgraph for each id_1.</h3>\n<pre><code>from torch_geometric.utils import k_hop_subgraph\n\nnodes = k_hop_subgraph(node_id, 2, pyg_G.edge_index, flow='target_to_source')[0]\nsubgraph = pyg_G.subgraph(nodes)\n</code></pre>\n<h3>4. Create a marker feature</h3>\n<p>Detailed inference and training code will be posted later.</p>",
          "votes": 2,
          "replies": []
        },
        {
          "id": 1861437,
          "author_name": "Y-Haneji",
          "author_url": "",
          "post_date": "2022-07-19T03:03:59.517000",
          "content": "<p><a href=\"https://www.kaggle.com/tubotubo\" target=\"_blank\">@tubotubo</a> <br>\nThank you for your explanation! <br>\nI'm looking forward to seeing your detailed code.</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 1857804,
      "author_name": "Rushabhmalode",
      "author_url": "",
      "post_date": "2022-07-16T12:09:20.610000",
      "content": "<p>It's a great solved on NLP and also advantages of GNN</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1856807,
      "author_name": "Bilal Suppal",
      "author_url": "",
      "post_date": "2022-07-15T16:02:04.360000",
      "content": "<p>One quesition, in stage4, on 2-hop-subgraph, seems only has the edge features in direct linked edge, how to deal with the others edge? Do you consider to share the inference code?</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1855801,
      "author_name": "Khizer Ismael",
      "author_url": "",
      "post_date": "2022-07-14T23:42:13.187000",
      "content": "<p>I like your approach very much. Thank you for sharing!</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1855216,
      "author_name": "Lucas See",
      "author_url": "",
      "post_date": "2022-07-14T11:20:57.167000",
      "content": "<p>Impressive and well-documented solution, thanks for taking the time to explain it!</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1852609,
      "author_name": "Naman Sharma",
      "author_url": "",
      "post_date": "2022-07-12T07:28:55.450000",
      "content": "<p>It's proper, well-structured, and organized work! 👍niceee</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1852604,
      "author_name": "Naman Sharma",
      "author_url": "",
      "post_date": "2022-07-12T07:26:56.217000",
      "content": "<p>Really detailed work</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1849955,
      "author_name": "The Devastator",
      "author_url": "",
      "post_date": "2022-07-10T00:39:49.450000",
      "content": "<p>Thanks for sharing your approach. It's really impressive and unique!</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1850622,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-07-10T14:45:48.737000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1934112,
      "author_name": "Mohammad Souri Laki",
      "author_url": "",
      "post_date": "2022-09-11T06:47:23.523000",
      "content": "<p>Great, thanks a lot</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1855865,
      "author_name": "Himanshu Goswami",
      "author_url": "",
      "post_date": "2022-07-15T01:49:02.633000",
      "content": "<p>Thanks for sharing!</p>",
      "votes": 0,
      "replies": []
    }
  ],
  "comment_count": 34,
  "messages": [
    {
      "id": 1849363,
      "postDate": "2022-07-09T12:56:45.730Z",
      "content": "<p>First of all, this competition would have been better without the leak.<br>\nIt is very sad that because of the leak, we can no longer distinguish between a good solution and a overfitting solution.</p>\n<h1>Summary</h1>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6958652%2F8709eae7598a8c4aa5f131aff0f0d128%2Fsolution.jpg?generation=1657372988563552&amp;alt=media\" alt=\"\"></p>\n<p>My solution consists of four stages</p>\n<ul>\n<li>Candidate generation<ul>\n<li>1st stage: Create candidates</li>\n<li>2nd stage: Filtering by LightGBM</li></ul></li>\n<li>Matching<ul>\n<li>3rd stage: match prediction by xlm-roberta-base and mdeberta-base-v3</li></ul></li>\n<li>PostProcessing<ul>\n<li>4th stage: Node Classification by GNN</li></ul></li>\n</ul>\n<h1>Cross validation strategy</h1>\n<p>GroupKFold(n_splits=2, group='point_of_interest')<br>\nThe CV score of my solution is 0.920</p>\n<h1>1st stage</h1>\n<ul>\n<li><p>Generates 60 candidates per id</p>\n<ul>\n<li>30 candidate neighborhoods per id using cosine similarity of text embedding by Tfidf<ul>\n<li>text: name, categories, address, city, state in a one sentence</li></ul></li>\n<li>30 candidate neighborhoods by haversine distance for per id</li></ul></li>\n</ul>\n<h1>2nd stage</h1>\n<ul>\n<li>Reduce to 3.7M pairs with high match probability using LightGBM</li>\n<li>Features<ul>\n<li>Missing value or not(name, cat)</li>\n<li>jaro(name, categories, address, city, state, zip, url, phone)</li>\n<li>leven(name, categories, address, city, state, zip, url, phone)</li>\n<li>haversine</li>\n<li>cos sim(tfidf embedding)</li>\n<li>mean top_k haversine, tfidf (k=5, 10, 15)</li></ul></li>\n<li>Using ForestInference</li>\n<li>Max IoU: 0.971</li>\n</ul>\n<h1>3rd stage</h1>\n<ul>\n<li>Predicts matche using a language model like DITTO</li>\n<li>model: xlm-roberta-base and mdeberta-base-v3</li>\n<li>Input<ul>\n<li>text: name_1 + [SEP] + name_2 + [COL] + categories_1 + [SEP] + categories_2 + [COL] + address_1 + [SEP] + address_2 + [COL] + city_1 + [SEP] + city_2 + [COL] + state_1 + [SEP] + state_2 + [COL] + zip_1 + [SEP] + zip_2</li>\n<li>numerical_feature</li>\n<li>haversine</li>\n<li>mean top_k haversine(k=5, 10, 15)</li></ul></li>\n<li>Training method<ul>\n<li>multitask learning: match and country (+0.003)</li>\n<li>augmentation</li></ul></li>\n</ul>\n<pre><code>import nlpaug as naw\n    aug_list = [\n  naw.RandomWordAug(action='swap', aug_p=0.1), \n  naw.RandomWordAug(action='delete', aug_p=0.1), \n  naw.SplitAug(aug_p=0.1)\n]\n</code></pre>\n<ul>\n<li>flip id_1, id_2</li>\n<li>TTA: flip id_1, id_2</li>\n</ul>\n<h1>4th stage</h1>\n<p>This stage is the most important part of my solution.<br>\nBy using GNN, both CV and LB are improved by about 0.02x.  </p>\n<p>Predicts the matching id for id_1 using GNN.</p>\n<ol>\n<li>Create a 2-hop-subgraph for each id_1(max iou: 0.993)  <br>\n<strong>example</strong> , Suppose the following table is given and we predict the id matching A.</li>\n</ol>\n<table>\n<thead>\n<tr>\n<th>id_1</th>\n<th>id_2</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>A</td>\n<td>B</td>\n</tr>\n<tr>\n<td>B</td>\n<td>A</td>\n</tr>\n<tr>\n<td>B</td>\n<td>C</td>\n</tr>\n<tr>\n<td>B</td>\n<td>D</td>\n</tr>\n<tr>\n<td>C</td>\n<td>D</td>\n</tr>\n<tr>\n<td>D</td>\n<td>B</td>\n</tr>\n<tr>\n<td>E</td>\n<td>A</td>\n</tr>\n<tr>\n<td>E</td>\n<td>F</td>\n</tr>\n<tr>\n<td>F</td>\n<td>E</td>\n</tr>\n</tbody>\n</table>\n<p>In this case, the 2-hop-subgraph for A is as follows(Think in terms of undirected graphs)</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6958652%2F26622119da7e573aa087ba1e4e942ff6%2FScreenshot%20from%202022-07-09%2021-30-51.png?generation=1657371037868405&amp;alt=media\" alt=\"\"></p>\n<ol>\n<li>Predict whether each node matches id_1</li>\n<li>Nodes that exceed the threshold are set as matches for id_1</li>\n</ol>\n<h2>Details</h2>\n<ul>\n<li>Model<br>\nThis model is based on the top solution of the Japanese competition platform atmaCup.</li>\n</ul>\n<pre><code>class SimpleGCN(nn.Module):\n    def __init__(self,\n                 num_node_features: int,\n                 num_edge_features: int,\n                #  num_classes: int,\n                 deg):\n        super(SimpleGCN, self).__init__()\n\n        aggregators = ['mean', 'min', 'max', 'std']\n        scalers = ['identity', 'amplification', 'attenuation']\n\n        towers = 4\n        pre_layer_num = 2\n        post_layer_num = 2\n        divide_input=False\n        self.convs = ModuleList()\n        self.batch_norms = ModuleList()\n        conv = PNAConv(in_channels=num_node_features, out_channels=32,\n                                    aggregators=aggregators, scalers=scalers, deg=deg,\n                                    edge_dim=num_edge_features, towers=towers, pre_layers=pre_layer_num, post_layers=post_layer_num,\n                                    divide_input=divide_input)\n        self.batch_norms.append(BatchNorm(32))\n        self.convs.append(conv)\n\n        for j in [32, 64, 128]:\n            conv = PNAConv(in_channels=j, out_channels=j*2, aggregators=aggregators, scalers=scalers, deg=deg, edge_dim=num_edge_features, towers=towers, pre_layers=pre_layer_num, post_layers=post_layer_num, divide_input=divide_input)\n            self.convs.append(conv)\n            self.batch_norms.append(BatchNorm(j*2))\n\n        self.dropout = nn.Dropout(0.2)\n        self.linear = nn.Linear(j*2, 1)\n\n    def forward(self, data):\n        x = data.x\n        edge_index = data.edge_index\n        edge_attr = data.edge_attr\n\n        for conv, batch_norm in zip(self.convs, self.batch_norms):\n            x = F.relu(batch_norm(conv(x, edge_index, edge_attr)))\n\n        x = self.dropout(x)\n        x = self.linear(x).flatten()\n        return x\n</code></pre>\n<ul>\n<li>node features<ul>\n<li>marker(I named it): 1 if the node corresponds to an id_1, 0 if it is a candidate.  <br>\nThe presence of this feature allows us to solve for node classification rather than link prediction.</li>\n<li>Missing value or not(name, categories, address, city, state, zip, url, phone)</li></ul></li>\n<li>edge features<ul>\n<li>mdeberta pred</li>\n<li>xlm-roberta pred</li>\n<li>lgbm pred</li>\n<li>jaro (address, city, state, zip, url, phone, categories)</li></ul></li>\n<li>loss function<ul>\n<li>IouLoss for each graph and the binary cross-entropy of the entire batch are the loss functions</li></ul></li>\n</ul>\n<pre><code>def iou_loss(logits, data):\n    sizes = degree(data.batch, dtype=torch.long).tolist()\n\n    logit_list = logits.sigmoid().split(sizes)\n    target_list = data.y.split(sizes)\n\n    loss = 0\n    for y_pred, y_target, group_size in zip(logit_list, target_list, data.group_size):\n\n        intersection = (y_pred * y_target).sum()\n        total = (y_pred + y_target).sum() + (group_size - y_target.sum())\n        union = total - intersection\n        loss += 1 - (intersection / union)\n\n    return loss / data.num_graphs\n\nloss = iou_loss + 0.1*BCELoss\n</code></pre>\n<h2>Advantages of GNN</h2>\n<ul>\n<li>Creating a 2-hop-subgraph greatly improved maxiou.</li>\n<li>Iou loss made it possible to optimize the competition metric directly.</li>\n<li>Fast inference</li>\n</ul>\n<h1>Public Score Timeline</h1>\n<ol>\n<li>20 candidates per id and mdeberta and lgbm stacking: 0.907</li>\n<li>60 candidates per id and xlm-roberta-base+mdeberta and lgbm stacking : 0.924</li>\n<li>60 candidates per id and xlm-roberta-base + mdeberta and GNN Post Processing: 0.946</li>\n</ol>",
      "rawMarkdown": "First of all, this competition would have been better without the leak.\nIt is very sad that because of the leak, we can no longer distinguish between a good solution and a overfitting solution.\n\n# Summary\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6958652%2F8709eae7598a8c4aa5f131aff0f0d128%2Fsolution.jpg?generation=1657372988563552&alt=media)\n\nMy solution consists of four stages\n\n- Candidate generation\n  - 1st stage: Create candidates\n  - 2nd stage: Filtering by LightGBM\n- Matching\n  - 3rd stage: match prediction by xlm-roberta-base and mdeberta-base-v3\n- PostProcessing\n  - 4th stage: Node Classification by GNN\n\n# Cross validation strategy\n\nGroupKFold(n_splits=2, group='point_of_interest')\nThe CV score of my solution is 0.920\n\n# 1st stage\n\n- Generates 60 candidates per id\n\n  - 30 candidate neighborhoods per id using cosine similarity of text embedding by Tfidf\n      - text: name, categories, address, city, state in a one sentence\n  - 30 candidate neighborhoods by haversine distance for per id\n\n# 2nd stage\n\n- Reduce to 3.7M pairs with high match probability using LightGBM\n- Features\n  - Missing value or not(name, cat)\n  - jaro(name, categories, address, city, state, zip, url, phone)\n  - leven(name, categories, address, city, state, zip, url, phone)\n  - haversine\n  - cos sim(tfidf embedding)\n  - mean top_k haversine, tfidf (k=5, 10, 15)\n- Using ForestInference\n- Max IoU: 0.971\n\n# 3rd stage\n\n- Predicts matche using a language model like DITTO\n- model: xlm-roberta-base and mdeberta-base-v3\n- Input\n  - text: name_1 + [SEP] + name_2 + [COL] + categories_1 + [SEP] + categories_2 + [COL] + address_1 + [SEP] + address_2 + [COL] + city_1 + [SEP] + city_2 + [COL] + state_1 + [SEP] + state_2 + [COL] + zip_1 + [SEP] + zip_2\n  - numerical_feature\n    - haversine\n    - mean top_k haversine(k=5, 10, 15)\n- Training method\n  - multitask learning: match and country (+0.003)\n  - augmentation\n```\nimport nlpaug as naw\n    aug_list = [\n  naw.RandomWordAug(action='swap', aug_p=0.1), \n  naw.RandomWordAug(action='delete', aug_p=0.1), \n  naw.SplitAug(aug_p=0.1)\n]\n```\n- flip id_1, id_2\n- TTA: flip id_1, id_2\n\n# 4th stage\n\nThis stage is the most important part of my solution.\nBy using GNN, both CV and LB are improved by about 0.02x.  \n\nPredicts the matching id for id_1 using GNN.\n\n1. Create a 2-hop-subgraph for each id_1(max iou: 0.993)  \n__example__ , Suppose the following table is given and we predict the id matching A.\n\n| id_1 | id_2 | \n| ---- | ---- | \n| A    | B    | \n| B    | A    | \n| B    | C    | \n| B    | D    | \n| C    | D    | \n| D    | B    | \n| E    | A    | \n| E    | F    | \n| F    | E    | \n\nIn this case, the 2-hop-subgraph for A is as follows(Think in terms of undirected graphs)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6958652%2F26622119da7e573aa087ba1e4e942ff6%2FScreenshot%20from%202022-07-09%2021-30-51.png?generation=1657371037868405&alt=media)\n\n2. Predict whether each node matches id_1\n3. Nodes that exceed the threshold are set as matches for id_1\n\n## Details\n\n- Model\nThis model is based on the top solution of the Japanese competition platform atmaCup.\n\n```\nclass SimpleGCN(nn.Module):\n    def __init__(self,\n                 num_node_features: int,\n                 num_edge_features: int,\n                #  num_classes: int,\n                 deg):\n        super(SimpleGCN, self).__init__()\n\n        aggregators = ['mean', 'min', 'max', 'std']\n        scalers = ['identity', 'amplification', 'attenuation']\n\n        towers = 4\n        pre_layer_num = 2\n        post_layer_num = 2\n        divide_input=False\n        self.convs = ModuleList()\n        self.batch_norms = ModuleList()\n        conv = PNAConv(in_channels=num_node_features, out_channels=32,\n                                    aggregators=aggregators, scalers=scalers, deg=deg,\n                                    edge_dim=num_edge_features, towers=towers, pre_layers=pre_layer_num, post_layers=post_layer_num,\n                                    divide_input=divide_input)\n        self.batch_norms.append(BatchNorm(32))\n        self.convs.append(conv)\n\n        for j in [32, 64, 128]:\n            conv = PNAConv(in_channels=j, out_channels=j*2, aggregators=aggregators, scalers=scalers, deg=deg, edge_dim=num_edge_features, towers=towers, pre_layers=pre_layer_num, post_layers=post_layer_num, divide_input=divide_input)\n            self.convs.append(conv)\n            self.batch_norms.append(BatchNorm(j*2))\n\n        self.dropout = nn.Dropout(0.2)\n        self.linear = nn.Linear(j*2, 1)\n\n    def forward(self, data):\n        x = data.x\n        edge_index = data.edge_index\n        edge_attr = data.edge_attr\n\n        for conv, batch_norm in zip(self.convs, self.batch_norms):\n            x = F.relu(batch_norm(conv(x, edge_index, edge_attr)))\n\n        x = self.dropout(x)\n        x = self.linear(x).flatten()\n        return x\n```\n\n- node features\n  - marker(I named it): 1 if the node corresponds to an id_1, 0 if it is a candidate.  \n  The presence of this feature allows us to solve for node classification rather than link prediction.\n  - Missing value or not(name, categories, address, city, state, zip, url, phone)\n- edge features\n  - mdeberta pred\n  - xlm-roberta pred\n  - lgbm pred\n  - jaro (address, city, state, zip, url, phone, categories)\n- loss function\n  - IouLoss for each graph and the binary cross-entropy of the entire batch are the loss functions\n\n```\ndef iou_loss(logits, data):\n    sizes = degree(data.batch, dtype=torch.long).tolist()\n\n    logit_list = logits.sigmoid().split(sizes)\n    target_list = data.y.split(sizes)\n\n    loss = 0\n    for y_pred, y_target, group_size in zip(logit_list, target_list, data.group_size):\n\n        intersection = (y_pred * y_target).sum()\n        total = (y_pred + y_target).sum() + (group_size - y_target.sum())\n        union = total - intersection\n        loss += 1 - (intersection / union)\n\n    return loss / data.num_graphs\n\nloss = iou_loss + 0.1*BCELoss\n```\n\n## Advantages of GNN\n  - Creating a 2-hop-subgraph greatly improved maxiou.\n  - Iou loss made it possible to optimize the competition metric directly.\n  - Fast inference\n\n# Public Score Timeline\n\n1. 20 candidates per id and mdeberta and lgbm stacking: 0.907\n2. 60 candidates per id and xlm-roberta-base+mdeberta and lgbm stacking : 0.924\n3. 60 candidates per id and xlm-roberta-base + mdeberta and GNN Post Processing: 0.946",
      "votes": 102
    },
    {
      "id": 1849980,
      "postDate": "2022-07-10T01:25:29.133Z",
      "content": "<p>Great share! Personally I like your approach more than the other solutions. </p>",
      "rawMarkdown": "Great share! Personally I like your approach more than the other solutions. ",
      "votes": 4
    },
    {
      "id": 1900809,
      "postDate": "2022-08-16T09:12:55.970Z",
      "content": "<p>Great share!  Could you share your code about train and inference? Thank you so much!</p>",
      "rawMarkdown": "Great share!  Could you share your code about train and inference? Thank you so much!",
      "votes": 2
    },
    {
      "id": 1850361,
      "postDate": "2022-07-10T10:54:19.257Z",
      "content": "<p>Greeeeeeate Work!  One quesition,   in stage4,  on 2-hop-subgraph,  seems only has the edge features in direct linked edge,  how to deal with the others edge?    Do you consider to share the inference code? </p>",
      "rawMarkdown": "Greeeeeeate Work!  One quesition,   in stage4,  on 2-hop-subgraph,  seems only has the edge features in direct linked edge,  how to deal with the others edge?    Do you consider to share the inference code? ",
      "votes": 2,
      "replies": [
        {
          "id": 1850374,
          "postDate": "2022-07-10T11:02:35.560Z",
          "content": "<p>Thanks for the question.</p>\n<p>2-hop-subgraph is created by considering the graph as an undirected graph.<br>\nThe implementation uses pytorch_geometric and can be written as follows<br>\n<a href=\"https://pytorch-geometric.readthedocs.io/en/latest/modules/utils.html#torch_geometric.utils.k_hop_subgraph\" target=\"_blank\">https://pytorch-geometric.readthedocs.io/en/latest/modules/utils.html#torch_geometric.utils.k_hop_subgraph</a></p>\n<pre><code>from torch_geometric.utils import k_hop_subgraph\n\nnodes = k_hop_subgraph(node_id, 2, G.edge_index, flow='target_to_source')[0]\nsubgraph = G.subgraph(nodes)\n</code></pre>\n<p>Sorry if this didn't answer your question.</p>",
          "rawMarkdown": "Thanks for the question.\n\n2-hop-subgraph is created by considering the graph as an undirected graph.\nThe implementation uses pytorch_geometric and can be written as follows\nhttps://pytorch-geometric.readthedocs.io/en/latest/modules/utils.html#torch_geometric.utils.k_hop_subgraph\n\n```\nfrom torch_geometric.utils import k_hop_subgraph\n\nnodes = k_hop_subgraph(node_id, 2, G.edge_index, flow='target_to_source')[0]\nsubgraph = G.subgraph(nodes)\n```\n\nSorry if this didn't answer your question."
        },
        {
          "id": 1850376,
          "postDate": "2022-07-10T11:05:10.483Z",
          "content": "<p>I'm thinking of sharing the inference code, but I'd like to organize it a bit before I share it with you.<br>\nThe current code is very dirty.😂</p>",
          "rawMarkdown": "I'm thinking of sharing the inference code, but I'd like to organize it a bit before I share it with you.\nThe current code is very dirty.😂",
          "votes": 4
        },
        {
          "id": 1852672,
          "postDate": "2022-07-12T08:33:11.683Z",
          "content": "<p><a href=\"https://www.kaggle.com/evilpsycho42\" target=\"_blank\">@evilpsycho42</a> </p>\n<blockquote>\n  <p>how to deal with the others edge?</p>\n</blockquote>\n<p>Since the author saids he/she deals with the target task as a node classification, I think we don't have to consider the relation between non-directly-connected nodes.</p>\n<blockquote>\n  <p>The presence of this feature allows us to solve for node classification rather than link prediction.</p>\n</blockquote>",
          "rawMarkdown": "@evilpsycho42 \n\n> how to deal with the others edge?\n\nSince the author saids he/she deals with the target task as a node classification, I think we don't have to consider the relation between non-directly-connected nodes.\n\n> The presence of this feature allows us to solve for node classification rather than link prediction."
        },
        {
          "id": 1852681,
          "postDate": "2022-07-12T08:45:54.383Z",
          "content": "<p><a href=\"https://www.kaggle.com/tubotubo\" target=\"_blank\">@tubotubo</a> I'm also interested in how the whole implementation (especially on GNN part) is done.<br>\nI don't care what the code looks like, the code would be very helpful.</p>\n<p>Thanks for sharing your solution.<br>\nI can't believe this all is done by only one person!</p>",
          "rawMarkdown": "@tubotubo I'm also interested in how the whole implementation (especially on GNN part) is done.\nI don't care what the code looks like, the code would be very helpful.\n\nThanks for sharing your solution.\nI can't believe this all is done by only one person!",
          "votes": 1
        },
        {
          "id": 1852725,
          "postDate": "2022-07-12T09:54:17.587Z",
          "content": "<p><a href=\"https://www.kaggle.com/tatamikenn\" target=\"_blank\">@tatamikenn</a> </p>\n<blockquote>\n  <p>how to deal with the others edge?</p>\n  <p>Since the author saids he/she deals with the target task as a node classification, I think we don't have to consider the relation between non-directly-connected nodes.</p>\n  <blockquote>\n    <p>The presence of this feature allows us to solve for node classification rather than link prediction.</p>\n  </blockquote>\n</blockquote>\n<p>Thank you for supplementing my explanation.</p>\n<blockquote>\n  <p>I'm also interested in how the whole implementation (especially on GNN part) is done.<br>\n  I don't care what the code looks like, the code would be very helpful.</p>\n  <p>Thanks for sharing your solution.<br>\n  I can't believe this all is done by only one person!</p>\n</blockquote>\n<p>Thank you for complimenting my solution!<br>\nI will release it to the public only after modifying the code.<br>\nHowever, since I am not sure when I will be able to release the code, I have shared it privately with you.</p>\n<p>I am glad that you are interested in my solution.</p>",
          "rawMarkdown": "@tatamikenn \n\n> how to deal with the others edge?\n> \n> Since the author saids he/she deals with the target task as a node classification, I think we don't have to consider the relation between non-directly-connected nodes.\n> \n> > The presence of this feature allows us to solve for node classification rather than link prediction.\n\nThank you for supplementing my explanation.\n\n> I'm also interested in how the whole implementation (especially on GNN part) is done.\n> I don't care what the code looks like, the code would be very helpful.\n\n> Thanks for sharing your solution.\n> I can't believe this all is done by only one person!\n\nThank you for complimenting my solution!\nI will release it to the public only after modifying the code.\nHowever, since I am not sure when I will be able to release the code, I have shared it privately with you.\n\nI am glad that you are interested in my solution.",
          "votes": 1
        }
      ]
    },
    {
      "id": 1850184,
      "postDate": "2022-07-10T07:19:18.147Z",
      "content": "<p>you did a good job👍</p>",
      "rawMarkdown": "you did a good job👍",
      "votes": 2
    },
    {
      "id": 1850037,
      "postDate": "2022-07-10T03:01:59.753Z",
      "content": "<p>Wow Great solution.  I am only your little version</p>",
      "rawMarkdown": "Wow Great solution.  I am only your little version",
      "votes": 2,
      "replies": [
        {
          "id": 1850040,
          "postDate": "2022-07-10T03:07:13.133Z",
          "content": "<p>Thank you. It is nice to be admired by a strong Grandmaster like you!</p>\n<p>I am very interested in your solution too.<br>\nDo you plan to share your solution?<br>\nCan you give me a brief explanation of your solution?</p>",
          "rawMarkdown": "Thank you. It is nice to be admired by a strong Grandmaster like you!\n\nI am very interested in your solution too.\nDo you plan to share your solution?\nCan you give me a brief explanation of your solution?",
          "votes": 2
        },
        {
          "id": 1850043,
          "postDate": "2022-07-10T03:13:21.763Z",
          "content": "<p>My solution only your stage 1+stage2+the last stage still Lgb😂. You did a great job. I feel regret for you because of the silly \"leak\"</p>",
          "rawMarkdown": "My solution only your stage 1+stage2+the last stage still Lgb😂. You did a great job. I feel regret for you because of the silly \"leak\"",
          "votes": 1
        },
        {
          "id": 1850045,
          "postDate": "2022-07-10T03:16:22.483Z",
          "content": "<p>Thank you for sharing your solution!<br>\nleak is very sad😭. I hope the second Foursquare competition will be held.</p>",
          "rawMarkdown": "Thank you for sharing your solution!\nleak is very sad😭. I hope the second Foursquare competition will be held.",
          "votes": 1
        }
      ]
    },
    {
      "id": 1858756,
      "postDate": "2022-07-17T07:22:29.573Z",
      "content": "<p>The information provided is so clear and easy to understand, Awesome post :)<br>\nLooking forward to see more !!!<br>\nlet's grow together</p>",
      "rawMarkdown": "The information provided is so clear and easy to understand, Awesome post :)\nLooking forward to see more !!!\nlet's grow together",
      "votes": 1
    },
    {
      "id": 1851508,
      "postDate": "2022-07-11T10:02:43.703Z",
      "content": "<p>I like your approach very much. Thank you for sharing!</p>",
      "rawMarkdown": "I like your approach very much. Thank you for sharing!",
      "votes": 1
    },
    {
      "id": 1850381,
      "postDate": "2022-07-10T11:10:52.553Z",
      "content": "<p>Thanks for sharing. GNN is really impressive!</p>",
      "rawMarkdown": "Thanks for sharing. GNN is really impressive!",
      "votes": 1
    },
    {
      "id": 1850340,
      "postDate": "2022-07-10T10:41:09.957Z",
      "content": "<p>Very impressive solution!</p>\n<p>thanks for share.</p>",
      "rawMarkdown": "Very impressive solution!\n\n thanks for share.",
      "votes": 1
    },
    {
      "id": 1849487,
      "postDate": "2022-07-09T15:13:02.020Z",
      "content": "<p>Your approach with GNNs is really impressive!<br>\nCongratulations!</p>",
      "rawMarkdown": "Your approach with GNNs is really impressive!\nCongratulations!",
      "votes": 1
    },
    {
      "id": 2592341,
      "postDate": "2024-01-08T13:56:39.017Z",
      "content": "<p>Thank you for sharing. Can you explain your approach for handling missing values, please?</p>",
      "rawMarkdown": "Thank you for sharing. Can you explain your approach for handling missing values, please?"
    },
    {
      "id": 2528194,
      "postDate": "2023-11-17T07:15:20.463Z",
      "content": "<p>Great share! I like your solution very much! :)</p>",
      "rawMarkdown": "Great share! I like your solution very much! :)"
    },
    {
      "id": 2081007,
      "postDate": "2022-12-30T18:10:57.567Z",
      "content": "<p>I have decided not to share the code because the data is no longer available.</p>",
      "rawMarkdown": "I have decided not to share the code because the data is no longer available."
    },
    {
      "id": 1861341,
      "postDate": "2022-07-19T01:26:41.157Z",
      "content": "<p><a href=\"https://www.kaggle.com/tubotubo\" target=\"_blank\">@tubotubo</a> <br>\nThank you for sharing a great solution, and I appreciate your new (meaning the version without leak) score and prize!<br>\nCould you give me some example of input objects to GNN or dataset implementation? Of course, the figure helped me a lot with understanding the idea, but I can't imagine how to convert the pair table to PyTorch dataset representing the graph…</p>",
      "rawMarkdown": "@tubotubo \nThank you for sharing a great solution, and I appreciate your new (meaning the version without leak) score and prize!\nCould you give me some example of input objects to GNN or dataset implementation? Of course, the figure helped me a lot with understanding the idea, but I can't imagine how to convert the pair table to PyTorch dataset representing the graph...",
      "replies": [
        {
          "id": 1861366,
          "postDate": "2022-07-19T01:54:50.367Z",
          "content": "<p><a href=\"https://www.kaggle.com/hanejiyuto\" target=\"_blank\">@hanejiyuto</a> <br>\nThank you for your question.<br>\nI have implemented GNN using <a href=\"https://pytorch-geometric.readthedocs.io/en/latest/index.html\" target=\"_blank\">pytorch_geometric</a> and <a href=\"https://networkx.org/\" target=\"_blank\">networkx</a>.</p>\n<p>Follow this procedure to create the inputs to the GNN.</p>\n<h3>1. Create a large graph in the following way.</h3>\n<p>df_pair</p>\n<table>\n<thead>\n<tr>\n<th>id_1</th>\n<th>id_2</th>\n<th>mdeberta</th>\n<th>xlm-roberta</th>\n<th>lgbm</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>A</td>\n<td>B</td>\n<td>0.9</td>\n<td>0.9</td>\n<td>0.4</td>\n</tr>\n<tr>\n<td>…</td>\n<td>…</td>\n<td>…</td>\n<td>…</td>\n<td>…</td>\n</tr>\n<tr>\n<td>Z</td>\n<td>A</td>\n<td>0.2</td>\n<td>0.5</td>\n<td>0.6</td>\n</tr>\n</tbody>\n</table>\n<pre><code>import networkx as nx\nfrom torch_geometric.utils.convert import from_networkx\n\ndef build_graph(df_pair, weight_att_cols):\n      nx_G = nx.from_pandas_edgelist(\n               df_pair, \n               source='id_1',\n               target='id_2',\n               edge_attr=weight_att_cols,\n               create_using=nx.DiGraph()\n      )\n\n      pyg_G = from_networkx(nx_G, all, all)\n\n      return pyg_G\n</code></pre>\n<h3>2. Partition a large graph into connected components</h3>\n<h3>3. Create a 2-hop-subgraph for each id_1.</h3>\n<pre><code>from torch_geometric.utils import k_hop_subgraph\n\nnodes = k_hop_subgraph(node_id, 2, pyg_G.edge_index, flow='target_to_source')[0]\nsubgraph = pyg_G.subgraph(nodes)\n</code></pre>\n<h3>4. Create a marker feature</h3>\n<p>Detailed inference and training code will be posted later.</p>",
          "rawMarkdown": "@hanejiyuto \nThank you for your question.\nI have implemented GNN using [pytorch_geometric](https://pytorch-geometric.readthedocs.io/en/latest/index.html) and [networkx](https://networkx.org/).\n\nFollow this procedure to create the inputs to the GNN.\n\n### 1. Create a large graph in the following way.\n\ndf_pair\n| id_1 | id_2 | mdeberta | xlm-roberta | lgbm | \n| ---- | ---- | -------- | ----------- | ---- | \n| A    | B    | 0.9      | 0.9         | 0.4  | \n| ...  | ...  | ...      | ...         | ...  | \n| Z    | A    | 0.2      | 0.5         | 0.6  | \n\n```\nimport networkx as nx\nfrom torch_geometric.utils.convert import from_networkx\n\ndef build_graph(df_pair, weight_att_cols):\n      nx_G = nx.from_pandas_edgelist(\n               df_pair, \n               source='id_1',\n               target='id_2',\n               edge_attr=weight_att_cols,\n               create_using=nx.DiGraph()\n      )\n    \n      pyg_G = from_networkx(nx_G, all, all)\n\n      return pyg_G\n```\n\n### 2. Partition a large graph into connected components\n\n### 3. Create a 2-hop-subgraph for each id_1.\n\n```\nfrom torch_geometric.utils import k_hop_subgraph\n\nnodes = k_hop_subgraph(node_id, 2, pyg_G.edge_index, flow='target_to_source')[0]\nsubgraph = pyg_G.subgraph(nodes)\n```\n\n### 4. Create a marker feature\n\n\nDetailed inference and training code will be posted later.",
          "votes": 2
        },
        {
          "id": 1861437,
          "postDate": "2022-07-19T03:03:59.517Z",
          "content": "<p><a href=\"https://www.kaggle.com/tubotubo\" target=\"_blank\">@tubotubo</a> <br>\nThank you for your explanation! <br>\nI'm looking forward to seeing your detailed code.</p>",
          "rawMarkdown": "@tubotubo \nThank you for your explanation! \nI'm looking forward to seeing your detailed code.",
          "votes": 1
        }
      ]
    },
    {
      "id": 1857804,
      "postDate": "2022-07-16T12:09:20.610Z",
      "content": "<p>It's a great solved on NLP and also advantages of GNN</p>",
      "rawMarkdown": "It's a great solved on NLP and also advantages of GNN"
    },
    {
      "id": 1856807,
      "postDate": "2022-07-15T16:02:04.360Z",
      "content": "<p>One quesition, in stage4, on 2-hop-subgraph, seems only has the edge features in direct linked edge, how to deal with the others edge? Do you consider to share the inference code?</p>",
      "rawMarkdown": " One quesition, in stage4, on 2-hop-subgraph, seems only has the edge features in direct linked edge, how to deal with the others edge? Do you consider to share the inference code?"
    },
    {
      "id": 1855801,
      "postDate": "2022-07-14T23:42:13.187Z",
      "content": "<p>I like your approach very much. Thank you for sharing!</p>",
      "rawMarkdown": "I like your approach very much. Thank you for sharing!"
    },
    {
      "id": 1855216,
      "postDate": "2022-07-14T11:20:57.167Z",
      "content": "<p>Impressive and well-documented solution, thanks for taking the time to explain it!</p>",
      "rawMarkdown": "Impressive and well-documented solution, thanks for taking the time to explain it!"
    },
    {
      "id": 1852609,
      "postDate": "2022-07-12T07:28:55.450Z",
      "content": "<p>It's proper, well-structured, and organized work! 👍niceee</p>",
      "rawMarkdown": "It's proper, well-structured, and organized work! 👍niceee"
    },
    {
      "id": 1852604,
      "postDate": "2022-07-12T07:26:56.217Z",
      "content": "<p>Really detailed work</p>",
      "rawMarkdown": "Really detailed work\n"
    },
    {
      "id": 1849955,
      "postDate": "2022-07-10T00:39:49.450Z",
      "content": "<p>Thanks for sharing your approach. It's really impressive and unique!</p>",
      "rawMarkdown": "Thanks for sharing your approach. It's really impressive and unique!\n"
    },
    {
      "id": 1850622,
      "postDate": "2022-07-10T14:45:48.737Z",
      "rawMarkdown": "",
      "isDeleted": true
    },
    {
      "id": 1934112,
      "postDate": "2022-09-11T06:47:23.523Z",
      "content": "<p>Great, thanks a lot</p>",
      "rawMarkdown": "Great, thanks a lot"
    },
    {
      "id": 1855865,
      "postDate": "2022-07-15T01:49:02.633Z",
      "content": "<p>Thanks for sharing!</p>",
      "rawMarkdown": "Thanks for sharing!"
    }
  ],
  "complete": true,
  "source": "live",
  "fetched_at": "2026-10-22T08:25:25Z",
  "competition_id": 35476,
  "competition_slug": "foursquare-location-matching",
  "team_id": 8481329,
  "solution_url": "/competitions/foursquare-location-matching/writeups/team-merge-master-13th-place-solution-gnn"
}