\n
![\"Screenshot-33\"](\"https://i.ibb.co/1RCRs8H/Screenshot-33.png\")
Some links mentioned about some bug in the public code. I personally didnt find anything as such & the codes were of exceptionally high quality. couple of minor observations I had was:
\nDid I miss anything else?
", "messages": [ { "id": "1896148", "postDate": "08/12/2022 16:10:26", "content": "Some links mentioned about some bug in the public code. I personally didnt find anything as such & the codes were of exceptionally high quality. couple of minor observations I had was:
\nDid I miss anything else?
", "rawMarkdown": "Some links mentioned about some bug in the public code. I personally didnt find anything as such & the codes were of exceptionally high quality. couple of minor observations I had was:\n- The calculation of the fts feature had a small typo but this did not make any diff to score\n- One observation which I felt could have an impact (but which didnt) was in the truncation of the EOS from the context code cells. For those code cells which are shorter than MAX_CODE_CELL_LENGTH, the EOS character is retained. So we end up with some records having EOS tokens in context some with none and since EOS is an imp token I thought this may make a big diff to scores but it didnt\n- Pairwise model where the last markdown (which is an imp markdown to score) does not have a corresponding pair (because no code cell comes after it) so it is almost always ranked incorrectly at inference. I handled this by not ensembling the first and last markdowns and relying solely on point wise model for these 2 markdown ranks. One other team handled it in a much more refined way by adding a dummy code cell on top and bottom (https://www.kaggle.com/competitions/AI4Code/discussion/343603)\n\nDid I miss anything else?", "votes": null }, { "id": "1896253", "postDate": "08/12/2022 17:40:08", "content": "There are a lot of bugs and some things, which can be changed to get potential improvement.
\nIf you look at the tokenization of the input sequences, you can see a lot of extra paddings between markdown and code cells, because of it the input sequences would have more potentially useful information but has themโฆ
\n
Fixing is very simple, you just need to comment these lines.
\n![\"Screenshot-1\"](\"https://i.ibb.co/FgXFGD2/Screenshot-1.png\")
The result is below.
\n![\"Screenshot-2\"](\"https://i.ibb.co/p0gGJ9N/Screenshot-2.png\")
I don't know how much did it boost, because we have a small amount of time, so we changed everything ๐.
\nWe changed correct_bias of optimizer to True, because it was offered by this article - https://arxiv.org/pdf/2006.05987.pdf
\n![\"Screenshot-3\"](\"https://i.ibb.co/RYXnX8K/Screenshot-3.png\")
![\"Screenshot-4\"](\"https://i.ibb.co/P1s9tZL/Screenshot-4.png\")
\n\nFixing is very simple, you just need to comment these lines.\n\n\nThe result is below.\n\n\n\nI don't know how much did it boost, because we have a small amount of time, so we changed everything ๐.\n\n\n\n\n",
"votes": null
}
],
"comments": [
{
"id": 1896253,
"author_name": "vad13irt",
"author_url": "",
"post_date": "08/12/2022 17:40:08",
"content": "There are a lot of bugs and some things, which can be changed to get potential improvement.
\nIf you look at the tokenization of the input sequences, you can see a lot of extra paddings between markdown and code cells, because of it the input sequences would have more potentially useful information but has themโฆ
\n
Fixing is very simple, you just need to comment these lines.
\n
The result is below.
\n
I don't know how much did it boost, because we have a small amount of time, so we changed everything ๐.
\nWe changed correct_bias of optimizer to True, because it was offered by this article - https://arxiv.org/pdf/2006.05987.pdf
\n
\n\nFixing is very simple, you just need to comment these lines.\n\n\nThe result is below.\n\n\n\nI don't know how much did it boost, because we have a small amount of time, so we changed everything ๐.\n\n\n\n\n"
},
"source": "meta"
}