![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1577135%2F3063e1ed8d95d5287ef7c49b0c4d6feb%2F48i7dw.jpg?generation=1660105813988975&alt=media\")
Since i probably won't have enough time for this competition, i decide to share my current solution: https://github.com/suicao/ai4code-baseline
\nMy inference notebook (0.837 LB): https://www.kaggle.com/code/suicaokhoailang/stronger-baseline-with-code-cells?scriptVersionId=96521306
\nThe idea is pretty simple, based on @aerdem4's baseline but we randomly sample up to 20 cells of code to use as a global context. Let me know if you have any questions and/or if the code isn't working.
\nHappy kaggling 😉
Since i probably won't have enough time for this competition, i decide to share my current solution: https://github.com/suicao/ai4code-baseline
\nMy inference notebook (0.837 LB): https://www.kaggle.com/code/suicaokhoailang/stronger-baseline-with-code-cells?scriptVersionId=96521306
\nThe idea is pretty simple, based on @aerdem4's baseline but we randomly sample up to 20 cells of code to use as a global context. Let me know if you have any questions and/or if the code isn't working.
\nHappy kaggling 😉
Thank you for sharing your solution.
\nThis is a nice starting point for the new commers.
Thank you for introducing a good solution.
\nI wonder how long it took to learn based on 1 epoch with kaggle notebook.
\nIt seems to take too long for resources here(kaggle).
", "rawMarkdown": "Thank you for introducing a good solution.\n\nI wonder how long it took to learn based on 1 epoch with kaggle notebook. \n\nIt seems to take too long for resources here(kaggle).", "votes": 1, "replies": [ { "id": 1802867, "postDate": "2022-05-27T08:47:56.350Z", "content": "It took me 9 hours on a 3080 for a single epoch, I don't think Kaggle's gpu is viable. You can train on a fraction of the data though, about 20% will give a comparable result.
", "rawMarkdown": "It took me 9 hours on a 3080 for a single epoch, I don't think Kaggle's gpu is viable. You can train on a fraction of the data though, about 20% will give a comparable result.", "votes": 5 }, { "id": 1804432, "postDate": "2022-05-29T00:49:17.207Z", "content": "Same here @heuingttiang . It takes lot of time on Kaggle for even simple competition. I generally end up training on colab. Now even that I am thinking to take monthly membership since I want to spend lot of time doing this.
", "rawMarkdown": "Same here @heuingttiang . It takes lot of time on Kaggle for even simple competition. I generally end up training on colab. Now even that I am thinking to take monthly membership since I want to spend lot of time doing this." } ] }, { "id": 1895393, "postDate": "2022-08-12T05:28:10.420Z", "content": "thank you for the kindly sharing. Actually, if you extend 20 to including more code cells such as 50, and do a simple ensembling, you can finally obtain a much higher (0.87~) result.
", "rawMarkdown": "thank you for the kindly sharing. Actually, if you extend 20 to including more code cells such as 50, and do a simple ensembling, you can finally obtain a much higher (0.87~) result." }, { "id": 1892261, "postDate": "2022-08-10T01:59:43.900Z", "content": "In my opinion, this is the best public notebook in this contest, thanks!
", "rawMarkdown": "In my opinion, this is the best public notebook in this contest, thanks!", "replies": [ { "id": 1892319, "postDate": "2022-08-10T03:27:27.897Z", "content": "Did it work?
", "rawMarkdown": "Did it work?" }, { "id": 1892330, "postDate": "2022-08-10T03:40:39.580Z", "content": "@suicaokhoailang yes, I think most of us used this model(or based on this model) as a key model.
", "rawMarkdown": "@suicaokhoailang yes, I think most of us used this model(or based on this model) as a key model." }, { "id": 1892370, "postDate": "2022-08-10T04:29:33.953Z", "content": "
I wonder why your kendall_tau evaluation:
\n 1 - 4 * total_inversions
\ninstead of host implementation:
\n 1 - 4 * total_inversions / total_2max, is this a bug or some tricks?
on the metrics.py file, this is what I have where did you get your code from ?
\ndef kendall_tau(ground_truth, predictions):\n total_inversions = 0\n total_2max = 0 # twice the maximum possible inversions across all instances\n for gt, pred in zip(ground_truth, predictions):\n ranks = [gt.index(x) for x in pred] # rank predicted order in terms of ground truth\n total_inversions += count_inversions(ranks)\n n = len(gt)\n total_2max += n * (n - 1)\n return 1 - 4 * total_inversions / total_2max\nHi, @suicaokhoailang thanks for sharing!!!
\nwhat was the batch size you used? for training and how much memory you have on your 3080?
", "rawMarkdown": "Hi, @suicaokhoailang thanks for sharing!!!\n\nwhat was the batch size you used? for training and how much memory you have on your 3080?\n" }, { "id": 1804624, "postDate": "2022-05-29T09:02:35.647Z", "content": "Nice one, you did some really good work here.
", "rawMarkdown": "Nice one, you did some really good work here." }, { "id": 1803723, "postDate": "2022-05-28T05:08:28.017Z", "content": "I have a question.
\nyou add the feature of code and md length in your training code:
if n_md + n_code == 0:\n fts = torch.FloatTensor([0])\n else:\n fts = torch.FloatTensor([n_md / (n_md + n_code)])\nWhat is your motivation to put this code in?
", "rawMarkdown": "I have a question.\nyou add the feature of code and md length in your training code:\n```\nif n_md + n_code == 0:\n fts = torch.FloatTensor([0])\n else:\n fts = torch.FloatTensor([n_md / (n_md + n_code)])\n```\nWhat is your motivation to put this code in?", "replies": [ { "id": 1803746, "postDate": "2022-05-28T05:58:05.500Z", "content": "Just some idea about using more global information that I haven't fully explored yet, I don't think that particular one helps.
", "rawMarkdown": "Just some idea about using more global information that I haven't fully explored yet, I don't think that particular one helps.", "votes": 1 }, { "id": 1803888, "postDate": "2022-05-28T09:26:26.770Z", "content": "Thank you for replying.
\nI understand it as part of trial and error.
I think there is something missing.
\nI keep getting this error when trying your solution
\nFileNotFoundError: [Errno 2] No such file or directory: '../input/ai4codemodelspublic/model.bin'
", "rawMarkdown": "I think there is something missing. \n\nI keep getting this error when trying your solution\n\nFileNotFoundError: [Errno 2] No such file or directory: '../input/ai4codemodelspublic/model.bin'", "replies": [ { "id": 1801603, "postDate": "2022-05-26T00:25:45.277Z", "content": "I didn't make my trained weights public to avoid flooding the leaderboard, you can always train it yourself.
", "rawMarkdown": "I didn't make my trained weights public to avoid flooding the leaderboard, you can always train it yourself.\n\n", "votes": 16 } ] }, { "id": 1800635, "postDate": "2022-05-25T05:32:59.203Z", "content": "Thanks for sharing!
\nHow much score boost did you get from adding fts?
I dont think it makes a difference, just an old experiment I forgot about.
", "rawMarkdown": "I dont think it makes a difference, just an old experiment I forgot about." } ] }, { "id": 1800721, "postDate": "2022-05-25T06:59:24.300Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1890498, "postDate": "2022-08-08T19:21:25.880Z", "content": "Thank You !!!
", "rawMarkdown": "Thank You !!!" }, { "id": 1882215, "postDate": "2022-08-03T06:18:34.620Z", "content": "Thanks for sharing!
", "rawMarkdown": "Thanks for sharing!" }, { "id": 1807190, "postDate": "2022-05-31T19:52:28.837Z", "content": "Excellent solution. Thanks for sharing!
", "rawMarkdown": "Excellent solution. Thanks for sharing!" } ], "comments": [ { "id": 1812413, "author_name": "aCode", "author_url": "", "post_date": "2022-06-05T19:06:44.923000", "content": "Thank you for sharing your solution.
\nThis is a nice starting point for the new commers.
Thank you for introducing a good solution.
\nI wonder how long it took to learn based on 1 epoch with kaggle notebook.
\nIt seems to take too long for resources here(kaggle).
", "votes": 1, "replies": [ { "id": 1802867, "author_name": "Khoi Nguyen", "author_url": "", "post_date": "2022-05-27T08:47:56.350000", "content": "It took me 9 hours on a 3080 for a single epoch, I don't think Kaggle's gpu is viable. You can train on a fraction of the data though, about 20% will give a comparable result.
", "votes": 5, "replies": [] }, { "id": 1804432, "author_name": "Chirag Desai", "author_url": "", "post_date": "2022-05-29T00:49:17.207000", "content": "Same here @heuingttiang . It takes lot of time on Kaggle for even simple competition. I generally end up training on colab. Now even that I am thinking to take monthly membership since I want to spend lot of time doing this.
", "votes": 0, "replies": [] } ] }, { "id": 1895393, "author_name": "xianchao wu", "author_url": "", "post_date": "2022-08-12T05:28:10.420000", "content": "thank you for the kindly sharing. Actually, if you extend 20 to including more code cells such as 50, and do a simple ensembling, you can finally obtain a much higher (0.87~) result.
", "votes": 0, "replies": [] }, { "id": 1892261, "author_name": "gezi", "author_url": "", "post_date": "2022-08-10T01:59:43.900000", "content": "In my opinion, this is the best public notebook in this contest, thanks!
", "votes": 0, "replies": [ { "id": 1892319, "author_name": "Khoi Nguyen", "author_url": "", "post_date": "2022-08-10T03:27:27.897000", "content": "Did it work?
", "votes": 0, "replies": [] }, { "id": 1892330, "author_name": "gezi", "author_url": "", "post_date": "2022-08-10T03:40:39.580000", "content": "@suicaokhoailang yes, I think most of us used this model(or based on this model) as a key model.
", "votes": 0, "replies": [] }, { "id": 1892370, "author_name": "Khoi Nguyen", "author_url": "", "post_date": "2022-08-10T04:29:33.953000", "content": "
I wonder why your kendall_tau evaluation:
\n 1 - 4 * total_inversions
\ninstead of host implementation:
\n 1 - 4 * total_inversions / total_2max, is this a bug or some tricks?
on the metrics.py file, this is what I have where did you get your code from ?
\ndef kendall_tau(ground_truth, predictions):\n total_inversions = 0\n total_2max = 0 # twice the maximum possible inversions across all instances\n for gt, pred in zip(ground_truth, predictions):\n ranks = [gt.index(x) for x in pred] # rank predicted order in terms of ground truth\n total_inversions += count_inversions(ranks)\n n = len(gt)\n total_2max += n * (n - 1)\n return 1 - 4 * total_inversions / total_2max\nHi, @suicaokhoailang thanks for sharing!!!
\nwhat was the batch size you used? for training and how much memory you have on your 3080?
", "votes": 0, "replies": [] }, { "id": 1804624, "author_name": "Chidiebere Chukwuemeka", "author_url": "", "post_date": "2022-05-29T09:02:35.647000", "content": "Nice one, you did some really good work here.
", "votes": 0, "replies": [] }, { "id": 1803723, "author_name": "furu-nag", "author_url": "", "post_date": "2022-05-28T05:08:28.017000", "content": "I have a question.
\nyou add the feature of code and md length in your training code:
if n_md + n_code == 0:\n fts = torch.FloatTensor([0])\n else:\n fts = torch.FloatTensor([n_md / (n_md + n_code)])\nWhat is your motivation to put this code in?
", "votes": 0, "replies": [ { "id": 1803746, "author_name": "Khoi Nguyen", "author_url": "", "post_date": "2022-05-28T05:58:05.500000", "content": "Just some idea about using more global information that I haven't fully explored yet, I don't think that particular one helps.
", "votes": 1, "replies": [] }, { "id": 1803888, "author_name": "furu-nag", "author_url": "", "post_date": "2022-05-28T09:26:26.770000", "content": "Thank you for replying.
\nI understand it as part of trial and error.
I think there is something missing.
\nI keep getting this error when trying your solution
\nFileNotFoundError: [Errno 2] No such file or directory: '../input/ai4codemodelspublic/model.bin'
", "votes": 0, "replies": [ { "id": 1801603, "author_name": "Khoi Nguyen", "author_url": "", "post_date": "2022-05-26T00:25:45.277000", "content": "I didn't make my trained weights public to avoid flooding the leaderboard, you can always train it yourself.
", "votes": 16, "replies": [] } ] }, { "id": 1800635, "author_name": "yuanzhe zhou", "author_url": "", "post_date": "2022-05-25T05:32:59.203000", "content": "Thanks for sharing!
\nHow much score boost did you get from adding fts?
I dont think it makes a difference, just an old experiment I forgot about.
", "votes": 0, "replies": [] } ] }, { "id": 1800721, "author_name": "", "author_url": "", "post_date": "2022-05-25T06:59:24.300000", "content": "", "votes": 0, "replies": [] }, { "id": 1890498, "author_name": "Gregory L.", "author_url": "", "post_date": "2022-08-08T19:21:25.880000", "content": "Thank You !!!
", "votes": 0, "replies": [] }, { "id": 1882215, "author_name": "blackBatty", "author_url": "", "post_date": "2022-08-03T06:18:34.620000", "content": "Thanks for sharing!
", "votes": 0, "replies": [] }, { "id": 1807190, "author_name": "Rubén Aguilera GarcÃa", "author_url": "", "post_date": "2022-05-31T19:52:28.837000", "content": "Excellent solution. Thanks for sharing!
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1800619": "Since i probably won't have enough time for this competition, i decide to share my current solution: https://github.com/suicao/ai4code-baseline\n\nMy inference notebook (0.837 LB): https://www.kaggle.com/code/suicaokhoailang/stronger-baseline-with-code-cells?scriptVersionId=96521306\n\nThe idea is pretty simple, based on @aerdem4's baseline but we randomly sample up to 20 cells of code to use as a global context. Let me know if you have any questions and/or if the code isn't working.\nHappy kaggling 😉", "1812413": "Thank you for sharing your solution.\nThis is a nice starting point for the new commers.", "1802860": "Thank you for introducing a good solution.\n\nI wonder how long it took to learn based on 1 epoch with kaggle notebook. \n\nIt seems to take too long for resources here(kaggle).", "1895393": "thank you for the kindly sharing. Actually, if you extend 20 to including more code cells such as 50, and do a simple ensembling, you can finally obtain a much higher (0.87~) result.", "1892261": "In my opinion, this is the best public notebook in this contest, thanks!", "1819966": "I wonder why your kendall_tau evaluation:\n `1 - 4 * total_inversions` \ninstead of host implementation:\n `1 - 4 * total_inversions / total_2max`, is this a bug or some tricks?", "1811486": "Hi, @suicaokhoailang thanks for sharing!!!\n\nwhat was the batch size you used? for training and how much memory you have on your 3080?\n", "1804624": "Nice one, you did some really good work here.", "1803723": "I have a question.\nyou add the feature of code and md length in your training code:\n```\nif n_md + n_code == 0:\n fts = torch.FloatTensor([0])\n else:\n fts = torch.FloatTensor([n_md / (n_md + n_code)])\n```\nWhat is your motivation to put this code in?", "1801562": "I think there is something missing. \n\nI keep getting this error when trying your solution\n\nFileNotFoundError: [Errno 2] No such file or directory: '../input/ai4codemodelspublic/model.bin'", "1800635": "Thanks for sharing! \nHow much score boost did you get from adding fts?", "1800721": "", "1890498": "Thank You !!!", "1882215": "Thanks for sharing!", "1807190": "Excellent solution. Thanks for sharing!" } }