{
  "id": 335818,
  "title": "15th place solution",
  "content": "<p>Although I was not able to win the gold medal this time, I would like to publish my solution.<br>\nPlease forgive me if my English is not correct because I used an automatic translation.<br>\nMy solution consists of Blocking, Matching, and Postprocessing.</p>\n<ul>\n<li><p>Blocking</p>\n<ul>\n<li>KNN in L2 norm for a matrix combining the following three (performed with three weights, Lat/Lng : Name : Categories = 3 : 10 : 1, 100 : 10 : 1, 3000 : 10 : 1)<ul>\n<li>Latitude and longitude</li>\n<li>Name (TFIDF)<ul>\n<li>After preprocessing by unidecode, vectorized by<ul>\n<li>TfidfVectorizer(analyzer='char_wb', ngram_range=(3, 4), sublinear_tf=True)</li></ul></li>\n<li>'char_wb' was better than 'word' with higher tolerance for misspellings.</li></ul></li>\n<li>Categories (MDS)<ul>\n<li>Using the matching rate between categories in the train data as distance, I created 4-dimensional embeddings using <a href=\"https://en.wikipedia.org/wiki/Multidimensional_scaling\" target=\"_blank\">Multi Dimensional Scaling (MDS)</a>.</li></ul></li></ul></li>\n<li>Exact matches for attributes (name, address, zip, phone (last 4 digits), url (domain name))</li>\n<li>Merged the above and aligned the id/near_id pairs in one direction and removed duplicates.</li>\n<li>12 candidates per 1 id (about 7M candidates on test data), max IOU w/o postprocessing is about 0.983.</li>\n<li>I also tried embedding based on deep learning, but it did not work well and took too much inference time, so I gave up on it.</li></ul></li>\n<li><p>Matching</p>\n<ul>\n<li>Distance and country tokens were added to the <a href=\"https://arxiv.org/abs/2004.00584\" target=\"_blank\">Ditto</a> based input and trained mdeberta-v3-base using fine tuning. Example inputs are as follows.<ul>\n<li>' [D36] [RU] [COL] name [VAL] Аптека [COL] categories [VAL] Pharmacies [SEP]  [COL] name [VAL] Аптека [COL] categories [VAL] Pharmacies [COL] address [VAL] Ул  Лесная д  8А',<br>\n' [D29] [HU] [COL] name [VAL] Kazánház [COL] categories [VAL] Bars [SEP]  [COL] name [VAL] Cézár Ház B Épület [COL] categories [VAL] Residential Buildings (Apartments / Condos) [COL] city [VAL] Budapest [COL] zip [VAL] 1132',<br>\n' [D16] [JP] [COL] name [VAL] スシロー [COL] categories [VAL] Sushi Restaurants [COL] address [VAL] 寺崎北1 7 4 [COL] city [VAL] 佐倉市 [COL] state [VAL] 千葉県 [COL] phone [VAL] 5830 [COL] zip [VAL] 285-0819 [COL] url [VAL] akindo-sushiro [SEP]  [COL] name [VAL] 幕張イオン',</li></ul></li>\n<li>Distance tokens are represented as [D0]-[D49]. [Di] means that the haversine distance is in the 2i to 2(i+1) percentile of the entire train data.</li>\n<li>The country token is simply the string of country.</li>\n<li>Fine tuning is done in 4 epochs, all positive pairs are used, negative pairs are divided into 4 parts and each part is assigned to each epoch.</li>\n<li>Trained by the entire train data, no validation. Single model, single fold.</li>\n<li>Google Colab (V100, high memory) was used, 40 hours per epoch (160 hours overall).</li>\n<li>Learning rate is 2e-5. Batch size is 36.</li>\n<li>Max token length is 128, which covers over 99% of train input sequences.</li>\n<li>Adversarial Training (FGM) was used (eps = 0.1) .</li></ul></li>\n<li><p>Postprocessing</p>\n<ul>\n<li>I constructed my own algorithm based on the group average method.<ul>\n<li>The usual group average method is too time-consuming due to the computational complexity of O(N^2) where N is the length of the test data. But this algorithm works with O(NKC^2), where K is the number of candidates per id, and C is the average cluster size.</li></ul></li>\n<li>The rough descriptions of the algorithm are as follows.  <ul>\n<li>Use a priority queue to process the clusters in order from the edge with the highest value of predictions. If the prediction is known for all possible edges between clusters, and the value retrieved from the priority queue matches the average of the predictions for all edges, merge the clusters with Union-Find. If there are edges for which the prediction has not yet been determined, the information is temporarily stored in the queue for additional predictions. When the additional prediction queue is full to some extent, the above matching process is used to obtain the missing predictions for the missing edges, calculating the average of the predictions for all edges, and the information is again stored in the priority queue.</li></ul></li>\n<li>The score was expected to be improved by roughly 0.025 compared to the case where postprocessing was not used.</li>\n<li>I initially used probability (0~1) as the value of prediction, but using the value of raw prediction before passing through sigmoid improved the score by about 0.002.</li>\n<li>The loop ends when the number of performing merging reaches 35% of the length of the test data.</li></ul></li>\n</ul>",
  "comments": [
    {
      "id": 1848712,
      "author_name": "The Devastator",
      "author_url": "",
      "post_date": "2022-07-08T20:53:09.797000",
      "content": "<p>Thanks for sharing your solution, and congrats. 15th is impressive even if it is not in the gold zone.</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1848220,
      "author_name": "nyanp",
      "author_url": "",
      "post_date": "2022-07-08T13:02:46.620000",
      "content": "<p>I used union-find based post-processing, but gave up on using group average because of the computational complexity. Great work!</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1847880,
      "author_name": "Psi",
      "author_url": "",
      "post_date": "2022-07-08T08:16:47.977000",
      "content": "<p>Nice pipeline, congrats!</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1847971,
      "author_name": "Vincent Schuler",
      "author_url": "",
      "post_date": "2022-07-08T09:32:57.693000",
      "content": "<p>Great work, thanks !</p>",
      "votes": 1,
      "replies": []
    }
  ],
  "comment_count": 4,
  "messages": [
    {
      "id": 1847548,
      "postDate": "2022-07-08T01:41:41.440Z",
      "content": "<p>Although I was not able to win the gold medal this time, I would like to publish my solution.<br>\nPlease forgive me if my English is not correct because I used an automatic translation.<br>\nMy solution consists of Blocking, Matching, and Postprocessing.</p>\n<ul>\n<li><p>Blocking</p>\n<ul>\n<li>KNN in L2 norm for a matrix combining the following three (performed with three weights, Lat/Lng : Name : Categories = 3 : 10 : 1, 100 : 10 : 1, 3000 : 10 : 1)<ul>\n<li>Latitude and longitude</li>\n<li>Name (TFIDF)<ul>\n<li>After preprocessing by unidecode, vectorized by<ul>\n<li>TfidfVectorizer(analyzer='char_wb', ngram_range=(3, 4), sublinear_tf=True)</li></ul></li>\n<li>'char_wb' was better than 'word' with higher tolerance for misspellings.</li></ul></li>\n<li>Categories (MDS)<ul>\n<li>Using the matching rate between categories in the train data as distance, I created 4-dimensional embeddings using <a href=\"https://en.wikipedia.org/wiki/Multidimensional_scaling\" target=\"_blank\">Multi Dimensional Scaling (MDS)</a>.</li></ul></li></ul></li>\n<li>Exact matches for attributes (name, address, zip, phone (last 4 digits), url (domain name))</li>\n<li>Merged the above and aligned the id/near_id pairs in one direction and removed duplicates.</li>\n<li>12 candidates per 1 id (about 7M candidates on test data), max IOU w/o postprocessing is about 0.983.</li>\n<li>I also tried embedding based on deep learning, but it did not work well and took too much inference time, so I gave up on it.</li></ul></li>\n<li><p>Matching</p>\n<ul>\n<li>Distance and country tokens were added to the <a href=\"https://arxiv.org/abs/2004.00584\" target=\"_blank\">Ditto</a> based input and trained mdeberta-v3-base using fine tuning. Example inputs are as follows.<ul>\n<li>' [D36] [RU] [COL] name [VAL] Аптека [COL] categories [VAL] Pharmacies [SEP]  [COL] name [VAL] Аптека [COL] categories [VAL] Pharmacies [COL] address [VAL] Ул  Лесная д  8А',<br>\n' [D29] [HU] [COL] name [VAL] Kazánház [COL] categories [VAL] Bars [SEP]  [COL] name [VAL] Cézár Ház B Épület [COL] categories [VAL] Residential Buildings (Apartments / Condos) [COL] city [VAL] Budapest [COL] zip [VAL] 1132',<br>\n' [D16] [JP] [COL] name [VAL] スシロー [COL] categories [VAL] Sushi Restaurants [COL] address [VAL] 寺崎北1 7 4 [COL] city [VAL] 佐倉市 [COL] state [VAL] 千葉県 [COL] phone [VAL] 5830 [COL] zip [VAL] 285-0819 [COL] url [VAL] akindo-sushiro [SEP]  [COL] name [VAL] 幕張イオン',</li></ul></li>\n<li>Distance tokens are represented as [D0]-[D49]. [Di] means that the haversine distance is in the 2i to 2(i+1) percentile of the entire train data.</li>\n<li>The country token is simply the string of country.</li>\n<li>Fine tuning is done in 4 epochs, all positive pairs are used, negative pairs are divided into 4 parts and each part is assigned to each epoch.</li>\n<li>Trained by the entire train data, no validation. Single model, single fold.</li>\n<li>Google Colab (V100, high memory) was used, 40 hours per epoch (160 hours overall).</li>\n<li>Learning rate is 2e-5. Batch size is 36.</li>\n<li>Max token length is 128, which covers over 99% of train input sequences.</li>\n<li>Adversarial Training (FGM) was used (eps = 0.1) .</li></ul></li>\n<li><p>Postprocessing</p>\n<ul>\n<li>I constructed my own algorithm based on the group average method.<ul>\n<li>The usual group average method is too time-consuming due to the computational complexity of O(N^2) where N is the length of the test data. But this algorithm works with O(NKC^2), where K is the number of candidates per id, and C is the average cluster size.</li></ul></li>\n<li>The rough descriptions of the algorithm are as follows.  <ul>\n<li>Use a priority queue to process the clusters in order from the edge with the highest value of predictions. If the prediction is known for all possible edges between clusters, and the value retrieved from the priority queue matches the average of the predictions for all edges, merge the clusters with Union-Find. If there are edges for which the prediction has not yet been determined, the information is temporarily stored in the queue for additional predictions. When the additional prediction queue is full to some extent, the above matching process is used to obtain the missing predictions for the missing edges, calculating the average of the predictions for all edges, and the information is again stored in the priority queue.</li></ul></li>\n<li>The score was expected to be improved by roughly 0.025 compared to the case where postprocessing was not used.</li>\n<li>I initially used probability (0~1) as the value of prediction, but using the value of raw prediction before passing through sigmoid improved the score by about 0.002.</li>\n<li>The loop ends when the number of performing merging reaches 35% of the length of the test data.</li></ul></li>\n</ul>",
      "rawMarkdown": "Although I was not able to win the gold medal this time, I would like to publish my solution.\nPlease forgive me if my English is not correct because I used an automatic translation.\nMy solution consists of Blocking, Matching, and Postprocessing.\n\n- Blocking\n\t- KNN in L2 norm for a matrix combining the following three (performed with three weights, Lat/Lng : Name : Categories = 3 : 10 : 1, 100 : 10 : 1, 3000 : 10 : 1)\n\t\t- Latitude and longitude\n\t\t- Name (TFIDF)\n\t\t\t- After preprocessing by unidecode, vectorized by\n\t\t\t  - TfidfVectorizer(analyzer='char_wb', ngram_range=(3, 4), sublinear_tf=True)\n\t\t\t- 'char_wb' was better than 'word' with higher tolerance for misspellings.\n\t\t- Categories (MDS)\n\t\t\t- Using the matching rate between categories in the train data as distance, I created 4-dimensional embeddings using [Multi Dimensional Scaling (MDS)](https://en.wikipedia.org/wiki/Multidimensional_scaling).\n\t- Exact matches for attributes (name, address, zip, phone (last 4 digits), url (domain name))\n\t- Merged the above and aligned the id/near_id pairs in one direction and removed duplicates.\n\t- 12 candidates per 1 id (about 7M candidates on test data), max IOU w/o postprocessing is about 0.983.\n\t- I also tried embedding based on deep learning, but it did not work well and took too much inference time, so I gave up on it.\n\n\n- Matching\n\t- Distance and country tokens were added to the [Ditto](https://arxiv.org/abs/2004.00584) based input and trained mdeberta-v3-base using fine tuning. Example inputs are as follows.\n\t  - ' [D36] [RU] [COL] name [VAL] Аптека [COL] categories [VAL] Pharmacies [SEP]  [COL] name [VAL] Аптека [COL] categories [VAL] Pharmacies [COL] address [VAL] Ул  Лесная д  8А',\n       ' [D29] [HU] [COL] name [VAL] Kazánház [COL] categories [VAL] Bars [SEP]  [COL] name [VAL] Cézár Ház B Épület [COL] categories [VAL] Residential Buildings (Apartments / Condos) [COL] city [VAL] Budapest [COL] zip [VAL] 1132',\n       ' [D16] [JP] [COL] name [VAL] スシロー [COL] categories [VAL] Sushi Restaurants [COL] address [VAL] 寺崎北1 7 4 [COL] city [VAL] 佐倉市 [COL] state [VAL] 千葉県 [COL] phone [VAL] 5830 [COL] zip [VAL] 285-0819 [COL] url [VAL] akindo-sushiro [SEP]  [COL] name [VAL] 幕張イオン',\n   - Distance tokens are represented as [D0]-[D49]. [Di] means that the haversine distance is in the 2i to 2(i+1) percentile of the entire train data.\n   - The country token is simply the string of country.\n   - Fine tuning is done in 4 epochs, all positive pairs are used, negative pairs are divided into 4 parts and each part is assigned to each epoch.\n   - Trained by the entire train data, no validation. Single model, single fold.\n   -  Google Colab (V100, high memory) was used, 40 hours per epoch (160 hours overall).\n   - Learning rate is 2e-5. Batch size is 36.\n   - Max token length is 128, which covers over 99% of train input sequences.\n   - Adversarial Training (FGM) was used (eps = 0.1) .\n\n\n- Postprocessing\n  - I constructed my own algorithm based on the group average method.\n      - The usual group average method is too time-consuming due to the computational complexity of O(N^2) where N is the length of the test data. But this algorithm works with O(NKC^2), where K is the number of candidates per id, and C is the average cluster size.\n  - The rough descriptions of the algorithm are as follows.  \n      - Use a priority queue to process the clusters in order from the edge with the highest value of predictions. If the prediction is known for all possible edges between clusters, and the value retrieved from the priority queue matches the average of the predictions for all edges, merge the clusters with Union-Find. If there are edges for which the prediction has not yet been determined, the information is temporarily stored in the queue for additional predictions. When the additional prediction queue is full to some extent, the above matching process is used to obtain the missing predictions for the missing edges, calculating the average of the predictions for all edges, and the information is again stored in the priority queue.\n  - The score was expected to be improved by roughly 0.025 compared to the case where postprocessing was not used.\n  - I initially used probability (0~1) as the value of prediction, but using the value of raw prediction before passing through sigmoid improved the score by about 0.002.\n  - The loop ends when the number of performing merging reaches 35% of the length of the test data.",
      "votes": 48
    },
    {
      "id": 1848712,
      "postDate": "2022-07-08T20:53:09.797Z",
      "content": "<p>Thanks for sharing your solution, and congrats. 15th is impressive even if it is not in the gold zone.</p>",
      "rawMarkdown": "Thanks for sharing your solution, and congrats. 15th is impressive even if it is not in the gold zone.\n",
      "votes": 1
    },
    {
      "id": 1848220,
      "postDate": "2022-07-08T13:02:46.620Z",
      "content": "<p>I used union-find based post-processing, but gave up on using group average because of the computational complexity. Great work!</p>",
      "rawMarkdown": "I used union-find based post-processing, but gave up on using group average because of the computational complexity. Great work!",
      "votes": 1
    },
    {
      "id": 1847880,
      "postDate": "2022-07-08T08:16:47.977Z",
      "content": "<p>Nice pipeline, congrats!</p>",
      "rawMarkdown": "Nice pipeline, congrats!",
      "votes": 1
    },
    {
      "id": 1847971,
      "postDate": "2022-07-08T09:32:57.693Z",
      "content": "<p>Great work, thanks !</p>",
      "rawMarkdown": "Great work, thanks !",
      "votes": 1
    }
  ],
  "complete": true,
  "source": "live",
  "fetched_at": "2026-10-22T08:25:01Z",
  "competition_id": 35476,
  "competition_slug": "foursquare-location-matching",
  "team_id": 8491306,
  "solution_url": "/competitions/foursquare-location-matching/writeups/shun-pi-15th-place-solution"
}