\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6004998%2F4999e4fdd47a9438d34c5440d8dfb7f7%2Fdownload.png?generation=1687510495717254&alt=media\")
Could anyone explain what is happening, whatever arch I use, all the models doing the same thing?
\nI'm using dropout and L1&L2, so even if it seems to be overfitting, the F1 score says another story.
the F1 is a validation one, not the training set
\n
Could anyone explain what is happening, whatever arch I use, all the models doing the same thing?
\nI'm using dropout and L1&L2, so even if it seems to be overfitting, the F1 score says another story.
the F1 is a validation one, not the training set
\n
The scale of your left hand graph only covers a small range, despite the visually striking nature of the curve. Your accuracy gets only a little worse (~0.935 to ~0.900). This could be compensated by a larger improvement in recall.
", "rawMarkdown": "The scale of your left hand graph only covers a small range, despite the visually striking nature of the curve. Your accuracy gets only a little worse (~0.935 to ~0.900). This could be compensated by a larger improvement in recall.", "votes": null }, { "id": "2314683", "postDate": "06/23/2023 14:43:36", "content": "how covering a small range with +100 epochs! I wonder if this behavior is because of the nature of the task, as overfitting the training data gives a conservative sense of FP
", "rawMarkdown": "how covering a small range with +100 epochs! I wonder if this behavior is because of the nature of the task, as overfitting the training data gives a conservative sense of FP", "votes": null }, { "id": "2319056", "postDate": "06/26/2023 19:48:18", "content": "I will repeat what @jbomitchell said but with a bit more elaboration. When loss values are relatively large and F1 values relatively small, we will have a strong inverse correlation between the two - any decrease in loss values will lead to an increase in F1. However, that doesn't hold all the way through the training. Once we get to the business end of training, this relationship breaks down and one can have them behaving in unexpected ways. Like John observed, a relatively small drop in accuracy (and a relatively small loss increase) could still lead to better F1 scores. And it isn't like F1 is steadily improving either.
", "rawMarkdown": "I will repeat what @jbomitchell said but with a bit more elaboration. When loss values are relatively large and F1 values relatively small, we will have a strong inverse correlation between the two - any decrease in loss values will lead to an increase in F1. However, that doesn't hold all the way through the training. Once we get to the business end of training, this relationship breaks down and one can have them behaving in unexpected ways. Like John observed, a relatively small drop in accuracy (and a relatively small loss increase) could still lead to better F1 scores. And it isn't like F1 is steadily improving either.", "votes": null }, { "id": "2319104", "postDate": "06/26/2023 21:20:29", "content": "thanks for your reply, yeah you are right I noted that a big change in f1 happens with a very small difference in loss or acc.
\nI tried different arch and combinations of regularization techniques and the model after overfitting reached f1 of 0.35 MAX
\nso it indicates that there are common data points, and this is the case of increasing F1 of Validation
so I may change the question to:
\nwhy L1 & L2 do not make an effect?
\nthe models are increasing weight Whatever I used different lamda, I know that those techniques make the model lose any weights that are not necessary, in other words,\" if the weight does not make a difference, then it's overfitting, so will make it zero(L1) or make it tends to zero(L2) \"
so what makes the model choose always to increase the weights, literally always, what is the difference between this task or these datasets?
", "rawMarkdown": "thanks for your reply, yeah you are right I noted that a big change in f1 happens with a very small difference in loss or acc. \nI tried different arch and combinations of regularization techniques and the model after overfitting reached f1 of **0.35 MAX** \nso it indicates that there are common data points, and this is the case of increasing F1 of Validation \n\nso I may change the question to:\nwhy L1 & L2 do not make an effect?\nthe models are increasing weight Whatever I used different lamda, I know that those techniques make the model lose any weights that are not necessary, in other words,\" if the weight does not make a difference, then it's overfitting, so will make it zero(L1) or make it tends to zero(L2) \"\n\nso what makes the model choose always to increase the weights, literally always, what is the difference between this task or these datasets?", "votes": null } ], "comments": [ { "id": 2314638, "author_name": "jbomitchell", "author_url": "", "post_date": "06/23/2023 13:59:50", "content": "The scale of your left hand graph only covers a small range, despite the visually striking nature of the curve. Your accuracy gets only a little worse (~0.935 to ~0.900). This could be compensated by a larger improvement in recall.
", "votes": null, "replies": [ { "id": 2314683, "author_name": "salaheldinelnabarawy", "author_url": "", "post_date": "06/23/2023 14:43:36", "content": "how covering a small range with +100 epochs! I wonder if this behavior is because of the nature of the task, as overfitting the training data gives a conservative sense of FP
", "votes": null, "replies": [] } ] }, { "id": 2319056, "author_name": "tilii7", "author_url": "", "post_date": "06/26/2023 19:48:18", "content": "I will repeat what @jbomitchell said but with a bit more elaboration. When loss values are relatively large and F1 values relatively small, we will have a strong inverse correlation between the two - any decrease in loss values will lead to an increase in F1. However, that doesn't hold all the way through the training. Once we get to the business end of training, this relationship breaks down and one can have them behaving in unexpected ways. Like John observed, a relatively small drop in accuracy (and a relatively small loss increase) could still lead to better F1 scores. And it isn't like F1 is steadily improving either.
", "votes": null, "replies": [ { "id": 2319104, "author_name": "salaheldinelnabarawy", "author_url": "", "post_date": "06/26/2023 21:20:29", "content": "thanks for your reply, yeah you are right I noted that a big change in f1 happens with a very small difference in loss or acc.
\nI tried different arch and combinations of regularization techniques and the model after overfitting reached f1 of 0.35 MAX
\nso it indicates that there are common data points, and this is the case of increasing F1 of Validation
so I may change the question to:
\nwhy L1 & L2 do not make an effect?
\nthe models are increasing weight Whatever I used different lamda, I know that those techniques make the model lose any weights that are not necessary, in other words,\" if the weight does not make a difference, then it's overfitting, so will make it zero(L1) or make it tends to zero(L2) \"
so what makes the model choose always to increase the weights, literally always, what is the difference between this task or these datasets?
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2314203": "Could anyone explain what is happening, whatever arch I use, all the models doing the same thing?\nI'm using dropout and L1&L2, so even if it seems to be overfitting, the F1 score says another story.\n\nthe F1 is a validation one, not the training set\n", "2314638": "The scale of your left hand graph only covers a small range, despite the visually striking nature of the curve. Your accuracy gets only a little worse (~0.935 to ~0.900). This could be compensated by a larger improvement in recall.", "2314683": "how covering a small range with +100 epochs! I wonder if this behavior is because of the nature of the task, as overfitting the training data gives a conservative sense of FP", "2319056": "I will repeat what @jbomitchell said but with a bit more elaboration. When loss values are relatively large and F1 values relatively small, we will have a strong inverse correlation between the two - any decrease in loss values will lead to an increase in F1. However, that doesn't hold all the way through the training. Once we get to the business end of training, this relationship breaks down and one can have them behaving in unexpected ways. Like John observed, a relatively small drop in accuracy (and a relatively small loss increase) could still lead to better F1 scores. And it isn't like F1 is steadily improving either.", "2319104": "thanks for your reply, yeah you are right I noted that a big change in f1 happens with a very small difference in loss or acc. \nI tried different arch and combinations of regularization techniques and the model after overfitting reached f1 of **0.35 MAX** \nso it indicates that there are common data points, and this is the case of increasing F1 of Validation \n\nso I may change the question to:\nwhy L1 & L2 do not make an effect?\nthe models are increasing weight Whatever I used different lamda, I know that those techniques make the model lose any weights that are not necessary, in other words,\" if the weight does not make a difference, then it's overfitting, so will make it zero(L1) or make it tends to zero(L2) \"\n\nso what makes the model choose always to increase the weights, literally always, what is the difference between this task or these datasets?" }, "source": "meta" }