{ "id": 415470, "title": "Update: 500k Graphs Data", "url": "/competitions/benetech-making-graphs-accessible/discussion/415470", "author_name": "", "post_date": "2023-06-06T15:09:32.956453400Z", "votes": 21, "comment_count": 17, "views": 0, "content": "

Due to the competition extension and external dataset license issues, I have decided to release another version of the 500k Generated Graphs.

\n

Here are some of the changes in the new version. (previous versions are all still available)

\n

Numerical Precision Change

\n

The biggest change is that the precision of numerical values is reduced. In my experiments I found that reducing the precision resulted in a higher score for end to end models like \"pix2struct\" and \"donut\". If you would like full precision data, use previous versions of the dataset.

\n

Training only on the generated data I was able to achieve an LB score of 0.59.

\n
 nr(y_true, y_pred):\n     sigmoid(( - np.clip(r2_score(y_true, y_pred), , ) ** .))\n\n sigmoid(x):\n      -  / ( + np.exp(-x))\n\n reduce_precision(arr):\n     i in range(-, ):\n        \n         = np.round(arr, decimals=i)\n         i <= :\n             = prec.astype(int)\n         = list(prec)\n        \n         nr(arr, prec) >= .:\n             prec\n     arr\n\n =\n(reduce_precision(before))\n
\n

Graph Conventions Change

\n

The following changes were made based on the graph conventions tab.

\n\n

Included Validation Data

\n

I also added a graphs_val folder which contains the extracted data provided by the competition organizers.

\n

Dataset Examples

\n

Here are some updated examples from the dataset. Note the changes in numerical precision.

\n

40 | 4 <0x0A> 41 | 4 <0x0A> 42 | 8 <0x0A> 43 | 6 <0x0A> 44 | 4 <0x0A> 45 | 9 <0x0A> 46 | 5 <0x0A> 47 | 7 <0x0A> 48 | 8 <0x0A> 49 | 6 <0x0A> 50 | 5 <0x0A> 51 | 4 <0x0A> 52 | 7
\n\"Dot

\n

Febuary | 230 <0x0A> March | 430 <0x0A> April | 280
\n\"Line

\n

0 | 4000 <0x0A> 0 | 4000 <0x0A> 1000 | 3000 <0x0A> 1000 | 3000 <0x0A> 1000 | 3000 <0x0A> 2000 | 3000 <0x0A> 2000 | 3000 <0x0A> 2000 | 3000 <0x0A> 3000 | 3000 <0x0A> 3000 | 3000 <0x0A> 3000 | 3000 <0x0A> 4000 | 3000 <0x0A> 4000 | 3000 <0x0A> 4000 | 3000 <0x0A> 4000 | 3000 <0x0A> 5000 | 2000 <0x0A> 5000 | 2000 <0x0A> 5000 | 2000 <0x0A> 6000 | 1000 <0x0A> 6000 | 1000 <0x0A> 6000 | 0
\n\"Scatter

", "messages": [ { "id": "2290120", "postDate": "06/06/2023 15:09:32", "content": "

Due to the competition extension and external dataset license issues, I have decided to release another version of the 500k Generated Graphs.

\n

Here are some of the changes in the new version. (previous versions are all still available)

\n

Numerical Precision Change

\n

The biggest change is that the precision of numerical values is reduced. In my experiments I found that reducing the precision resulted in a higher score for end to end models like \"pix2struct\" and \"donut\". If you would like full precision data, use previous versions of the dataset.

\n

Training only on the generated data I was able to achieve an LB score of 0.59.

\n
 nr(y_true, y_pred):\n     sigmoid(( - np.clip(r2_score(y_true, y_pred), , ) ** .))\n\n sigmoid(x):\n      -  / ( + np.exp(-x))\n\n reduce_precision(arr):\n     i in range(-, ):\n        \n         = np.round(arr, decimals=i)\n         i <= :\n             = prec.astype(int)\n         = list(prec)\n        \n         nr(arr, prec) >= .:\n             prec\n     arr\n\n =\n(reduce_precision(before))\n
\n

Graph Conventions Change

\n

The following changes were made based on the graph conventions tab.

\n\n

Included Validation Data

\n

I also added a graphs_val folder which contains the extracted data provided by the competition organizers.

\n

Dataset Examples

\n

Here are some updated examples from the dataset. Note the changes in numerical precision.

\n

40 | 4 <0x0A> 41 | 4 <0x0A> 42 | 8 <0x0A> 43 | 6 <0x0A> 44 | 4 <0x0A> 45 | 9 <0x0A> 46 | 5 <0x0A> 47 | 7 <0x0A> 48 | 8 <0x0A> 49 | 6 <0x0A> 50 | 5 <0x0A> 51 | 4 <0x0A> 52 | 7
\n\"Dot

\n

Febuary | 230 <0x0A> March | 430 <0x0A> April | 280
\n\"Line

\n

0 | 4000 <0x0A> 0 | 4000 <0x0A> 1000 | 3000 <0x0A> 1000 | 3000 <0x0A> 1000 | 3000 <0x0A> 2000 | 3000 <0x0A> 2000 | 3000 <0x0A> 2000 | 3000 <0x0A> 3000 | 3000 <0x0A> 3000 | 3000 <0x0A> 3000 | 3000 <0x0A> 4000 | 3000 <0x0A> 4000 | 3000 <0x0A> 4000 | 3000 <0x0A> 4000 | 3000 <0x0A> 5000 | 2000 <0x0A> 5000 | 2000 <0x0A> 5000 | 2000 <0x0A> 6000 | 1000 <0x0A> 6000 | 1000 <0x0A> 6000 | 0
\n\"Scatter

", "rawMarkdown": "Due to the competition extension and external dataset license issues, I have decided to release another version of the [500k Generated Graphs](https://www.kaggle.com/datasets/brendanartley/benetech-extra-generated-data). \n\nHere are some of the changes in the new version. (previous versions are all still available) \n\n## Numerical Precision Change \n\nThe biggest change is that the precision of numerical values is reduced. In my experiments I found that reducing the precision resulted in a higher score for end to end models like \"pix2struct\" and \"donut\". If you would like full precision data, use previous versions of the dataset.\n\nTraining **only** on the generated data I was able to achieve an LB score of 0.59.\n\n```\ndef nr(y_true, y_pred):\n return sigmoid((1 - np.clip(r2_score(y_true, y_pred), 0, 1) ** 0.5))\n\ndef sigmoid(x):\n return 2 - 2 / (1 + np.exp(-x))\n\ndef reduce_precision(arr):\n for i in range(-4, 2):\n # Round array\n prec = np.round(arr, decimals=i)\n if i <= 0:\n prec = prec.astype(int)\n prec = list(prec)\n # Check if nrmse is close enough\n if nr(arr, prec) >= 0.98:\n return prec\n return arr\n\nbefore = [1234, 2345, 1232, 1276, 2031]\nprint(reduce_precision(before))\n[1200, 2300, 1200, 1300, 2000]\n```\n\n## Graph Conventions Change\n\nThe following changes were made based on the [graph conventions tab](https://www.kaggle.com/competitions/benetech-making-graphs-accessible/overview/graph-conventions).\n\n- Updated # data points based on distribution of extracted data\n- Increased # of words in categoricals\n- Increased # of words in titles/subtitles\n- Added \"10 to 20, 30 to 40\" style for categoricals\n- Axis margin edits\n- Line: All x-tick labels do not have a value\n- Scatter: Added scatters that look similar to line plots\n- Dot: Randomly removed 1/2 of numerical x-axis labels\n\n## Included Validation Data\n\nI also added a `graphs_val` folder which contains the extracted data provided by the competition organizers.\n\n## Dataset Examples\n\nHere are some updated examples from the dataset. Note the changes in numerical precision.\n\n`40 | 4 <0x0A> 41 | 4 <0x0A> 42 | 8 <0x0A> 43 | 6 <0x0A> 44 | 4 <0x0A> 45 | 9 <0x0A> 46 | 5 <0x0A> 47 | 7 <0x0A> 48 | 8 <0x0A> 49 | 6 <0x0A> 50 | 5 <0x0A> 51 | 4 <0x0A> 52 | 7`\n![Dot Plot](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5570735%2Ff9d02a821c272e234694246675d07b6f%2Fd.jpg?generation=1686063794556412&alt=media)\n\n`Febuary | 230 <0x0A> March | 430 <0x0A> April | 280`\n![Line Plot](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5570735%2F0900cc2ae861606ae957f2c3138c23b5%2Fl.jpg?generation=1686063890184764&alt=media)\n\n`0 | 4000 <0x0A> 0 | 4000 <0x0A> 1000 | 3000 <0x0A> 1000 | 3000 <0x0A> 1000 | 3000 <0x0A> 2000 | 3000 <0x0A> 2000 | 3000 <0x0A> 2000 | 3000 <0x0A> 3000 | 3000 <0x0A> 3000 | 3000 <0x0A> 3000 | 3000 <0x0A> 4000 | 3000 <0x0A> 4000 | 3000 <0x0A> 4000 | 3000 <0x0A> 4000 | 3000 <0x0A> 5000 | 2000 <0x0A> 5000 | 2000 <0x0A> 5000 | 2000 <0x0A> 6000 | 1000 <0x0A> 6000 | 1000 <0x0A> 6000 | 0`\n![Scatter plot](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5570735%2F65ef5cbd4480839455ff150e5bc0817d%2Fs.jpg?generation=1686064090717212&alt=media)", "votes": null }, { "id": "2290175", "postDate": "06/06/2023 15:51:19", "content": "

Thank you for your continued effort on this data, after these competition changes I would expect this to become even more valuable.

", "rawMarkdown": "Thank you for your continued effort on this data, after these competition changes I would expect this to become even more valuable.", "votes": null }, { "id": "2290505", "postDate": "06/06/2023 21:45:26", "content": "

Thank you for your efforts!

", "rawMarkdown": "Thank you for your efforts!", "votes": null }, { "id": "2290665", "postDate": "06/07/2023 03:07:27", "content": "

Good job. @brendanartley How can these generated images be utilized? Can they be used as pre-trained data or be trained in conjunction with competition training data?

", "rawMarkdown": "Good job. @brendanartley How can these generated images be utilized? Can they be used as pre-trained data or be trained in conjunction with competition training data?", "votes": null }, { "id": "2291372", "postDate": "06/07/2023 14:09:26", "content": "

Either could be a good strategy.

\n

I'm not sure how much to oversample the extracted data though as the private LB (61% of the score) comes from a different source. Data Splits.

", "rawMarkdown": "Either could be a good strategy.\n\nI'm not sure how much to oversample the extracted data though as the private LB (61% of the score) comes from a different source. [Data Splits](https://www.kaggle.com/competitions/benetech-making-graphs-accessible/data).", "votes": null }, { "id": "2291438", "postDate": "06/07/2023 14:49:45", "content": "

Have you measured how much of the data you generated contributes to the final performance of the model, for example 100k, 200k, 300k..?

", "rawMarkdown": "Have you measured how much of the data you generated contributes to the final performance of the model, for example 100k, 200k, 300k..?", "votes": null }, { "id": "2291496", "postDate": "06/07/2023 15:34:05", "content": "

I'm not sure I understand the question. Currently I train on all 500k images for 1 epoch.

", "rawMarkdown": "I'm not sure I understand the question. Currently I train on all 500k images for 1 epoch.", "votes": null }, { "id": "2291515", "postDate": "06/07/2023 15:44:18", "content": "

One additional question: 500k images with 1 epoch => .59 is that correct?

", "rawMarkdown": "One additional question: 500k images with 1 epoch => .59 is that correct?", "votes": null }, { "id": "2291521", "postDate": "06/07/2023 15:48:53", "content": "

If I would do it, I would go with the pretraining with generated + competition and then fine-tune on only competition data. @marbury

", "rawMarkdown": "If I would do it, I would go with the pretraining with generated + competition and then fine-tune on only competition data. @marbury", "votes": null }, { "id": "2291547", "postDate": "06/07/2023 16:05:02", "content": "

Yeah, that is correct.

", "rawMarkdown": "Yeah, that is correct.", "votes": null }, { "id": "2293262", "postDate": "06/09/2023 04:16:39", "content": "

May I ask the max_patches you are using?

", "rawMarkdown": "May I ask the max_patches you are using?", "votes": null }, { "id": "2293429", "postDate": "06/09/2023 07:21:24", "content": "

Hi,@brendanartley,thank you for your data. When I use these additional data for training, loss is very unstable compared to using only the original data.Is this normal?

", "rawMarkdown": "Hi,[@brendanartley](https://www.kaggle.com/brendanartley),thank you for your data. When I use these additional data for training, loss is very unstable compared to using only the original data.Is this normal?", "votes": null }, { "id": "2293812", "postDate": "06/09/2023 14:24:29", "content": "

Max patches is set to 1024. Training loss is stable for me.

", "rawMarkdown": "Max patches is set to 1024. Training loss is stable for me.", "votes": null }, { "id": "2293929", "postDate": "06/09/2023 15:28:03", "content": "

I mean suppose we use 1M generated images, will the performance of model be better?

", "rawMarkdown": "I mean suppose we use 1M generated images, will the performance of model be better?", "votes": null }, { "id": "2294381", "postDate": "06/10/2023 03:01:36", "content": "

Models need sufficient information to learn patterns. Once you start adding new images it will reach a point where adding more images won't add any variance. The model will have learned all the patterns it can. What I mean it's not only about the number of images you train with but also the variance in data. What you can expect is that if you add some couple thousands images you will see a boost in performance but later adding more images will just produce a very little boost. This relates well with the law of diminishing marginal returns. @marbury

", "rawMarkdown": "Models need sufficient information to learn patterns. Once you start adding new images it will reach a point where adding more images won't add any variance. The model will have learned all the patterns it can. What I mean it's not only about the number of images you train with but also the variance in data. What you can expect is that if you add some couple thousands images you will see a boost in performance but later adding more images will just produce a very little boost. This relates well with the law of diminishing marginal returns. @marbury", "votes": null }, { "id": "2294847", "postDate": "06/10/2023 11:35:58", "content": "

hi what's r2_score meaning in nr function

", "rawMarkdown": "hi what's r2_score meaning in nr function", "votes": null }, { "id": "2295014", "postDate": "06/10/2023 14:15:26", "content": "

The nr() function is just shorthand for normalized_rmse(). I used functions which come from the Competition Metric - Benetech Mixed-Match Notebook.

", "rawMarkdown": "The `nr()` function is just shorthand for `normalized_rmse()`. I used functions which come from the [Competition Metric - Benetech Mixed-Match Notebook](https://www.kaggle.com/code/ryanholbrook/competition-metric-benetech-mixed-match).", "votes": null }, { "id": "2295498", "postDate": "06/11/2023 03:37:34", "content": "

thx so much

", "rawMarkdown": "thx so much", "votes": null } ], "comments": [ { "id": 2290175, "author_name": "cody11null", "author_url": "", "post_date": "06/06/2023 15:51:19", "content": "

Thank you for your continued effort on this data, after these competition changes I would expect this to become even more valuable.

", "votes": null, "replies": [] }, { "id": 2290505, "author_name": "asarvazyan", "author_url": "", "post_date": "06/06/2023 21:45:26", "content": "

Thank you for your efforts!

", "votes": null, "replies": [] }, { "id": 2290665, "author_name": "zifencai", "author_url": "", "post_date": "06/07/2023 03:07:27", "content": "

Good job. @brendanartley How can these generated images be utilized? Can they be used as pre-trained data or be trained in conjunction with competition training data?

", "votes": null, "replies": [ { "id": 2291372, "author_name": "brendanartley", "author_url": "", "post_date": "06/07/2023 14:09:26", "content": "

Either could be a good strategy.

\n

I'm not sure how much to oversample the extracted data though as the private LB (61% of the score) comes from a different source. Data Splits.

", "votes": null, "replies": [ { "id": 2291438, "author_name": "zifencai", "author_url": "", "post_date": "06/07/2023 14:49:45", "content": "

Have you measured how much of the data you generated contributes to the final performance of the model, for example 100k, 200k, 300k..?

", "votes": null, "replies": [ { "id": 2291496, "author_name": "brendanartley", "author_url": "", "post_date": "06/07/2023 15:34:05", "content": "

I'm not sure I understand the question. Currently I train on all 500k images for 1 epoch.

", "votes": null, "replies": [ { "id": 2291515, "author_name": "snnclsr", "author_url": "", "post_date": "06/07/2023 15:44:18", "content": "

One additional question: 500k images with 1 epoch => .59 is that correct?

", "votes": null, "replies": [ { "id": 2291547, "author_name": "brendanartley", "author_url": "", "post_date": "06/07/2023 16:05:02", "content": "

Yeah, that is correct.

", "votes": null, "replies": [ { "id": 2293262, "author_name": "sandeepmnair", "author_url": "", "post_date": "06/09/2023 04:16:39", "content": "

May I ask the max_patches you are using?

", "votes": null, "replies": [ { "id": 2293812, "author_name": "brendanartley", "author_url": "", "post_date": "06/09/2023 14:24:29", "content": "

Max patches is set to 1024. Training loss is stable for me.

", "votes": null, "replies": [] } ] }, { "id": 2293429, "author_name": "areno9516", "author_url": "", "post_date": "06/09/2023 07:21:24", "content": "

Hi,@brendanartley,thank you for your data. When I use these additional data for training, loss is very unstable compared to using only the original data.Is this normal?

", "votes": null, "replies": [] } ] } ] }, { "id": 2293929, "author_name": "zifencai", "author_url": "", "post_date": "06/09/2023 15:28:03", "content": "

I mean suppose we use 1M generated images, will the performance of model be better?

", "votes": null, "replies": [ { "id": 2294381, "author_name": "alejopaullier", "author_url": "", "post_date": "06/10/2023 03:01:36", "content": "

Models need sufficient information to learn patterns. Once you start adding new images it will reach a point where adding more images won't add any variance. The model will have learned all the patterns it can. What I mean it's not only about the number of images you train with but also the variance in data. What you can expect is that if you add some couple thousands images you will see a boost in performance but later adding more images will just produce a very little boost. This relates well with the law of diminishing marginal returns. @marbury

", "votes": null, "replies": [] } ] } ] } ] } ] }, { "id": 2291521, "author_name": "snnclsr", "author_url": "", "post_date": "06/07/2023 15:48:53", "content": "

If I would do it, I would go with the pretraining with generated + competition and then fine-tune on only competition data. @marbury

", "votes": null, "replies": [] } ] }, { "id": 2294847, "author_name": "leehann", "author_url": "", "post_date": "06/10/2023 11:35:58", "content": "

hi what's r2_score meaning in nr function

", "votes": null, "replies": [ { "id": 2295014, "author_name": "brendanartley", "author_url": "", "post_date": "06/10/2023 14:15:26", "content": "

The nr() function is just shorthand for normalized_rmse(). I used functions which come from the Competition Metric - Benetech Mixed-Match Notebook.

", "votes": null, "replies": [ { "id": 2295498, "author_name": "leehann", "author_url": "", "post_date": "06/11/2023 03:37:34", "content": "

thx so much

", "votes": null, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2290120": "Due to the competition extension and external dataset license issues, I have decided to release another version of the [500k Generated Graphs](https://www.kaggle.com/datasets/brendanartley/benetech-extra-generated-data). \n\nHere are some of the changes in the new version. (previous versions are all still available) \n\n## Numerical Precision Change \n\nThe biggest change is that the precision of numerical values is reduced. In my experiments I found that reducing the precision resulted in a higher score for end to end models like \"pix2struct\" and \"donut\". If you would like full precision data, use previous versions of the dataset.\n\nTraining **only** on the generated data I was able to achieve an LB score of 0.59.\n\n```\ndef nr(y_true, y_pred):\n return sigmoid((1 - np.clip(r2_score(y_true, y_pred), 0, 1) ** 0.5))\n\ndef sigmoid(x):\n return 2 - 2 / (1 + np.exp(-x))\n\ndef reduce_precision(arr):\n for i in range(-4, 2):\n # Round array\n prec = np.round(arr, decimals=i)\n if i <= 0:\n prec = prec.astype(int)\n prec = list(prec)\n # Check if nrmse is close enough\n if nr(arr, prec) >= 0.98:\n return prec\n return arr\n\nbefore = [1234, 2345, 1232, 1276, 2031]\nprint(reduce_precision(before))\n[1200, 2300, 1200, 1300, 2000]\n```\n\n## Graph Conventions Change\n\nThe following changes were made based on the [graph conventions tab](https://www.kaggle.com/competitions/benetech-making-graphs-accessible/overview/graph-conventions).\n\n- Updated # data points based on distribution of extracted data\n- Increased # of words in categoricals\n- Increased # of words in titles/subtitles\n- Added \"10 to 20, 30 to 40\" style for categoricals\n- Axis margin edits\n- Line: All x-tick labels do not have a value\n- Scatter: Added scatters that look similar to line plots\n- Dot: Randomly removed 1/2 of numerical x-axis labels\n\n## Included Validation Data\n\nI also added a `graphs_val` folder which contains the extracted data provided by the competition organizers.\n\n## Dataset Examples\n\nHere are some updated examples from the dataset. Note the changes in numerical precision.\n\n`40 | 4 <0x0A> 41 | 4 <0x0A> 42 | 8 <0x0A> 43 | 6 <0x0A> 44 | 4 <0x0A> 45 | 9 <0x0A> 46 | 5 <0x0A> 47 | 7 <0x0A> 48 | 8 <0x0A> 49 | 6 <0x0A> 50 | 5 <0x0A> 51 | 4 <0x0A> 52 | 7`\n![Dot Plot](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5570735%2Ff9d02a821c272e234694246675d07b6f%2Fd.jpg?generation=1686063794556412&alt=media)\n\n`Febuary | 230 <0x0A> March | 430 <0x0A> April | 280`\n![Line Plot](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5570735%2F0900cc2ae861606ae957f2c3138c23b5%2Fl.jpg?generation=1686063890184764&alt=media)\n\n`0 | 4000 <0x0A> 0 | 4000 <0x0A> 1000 | 3000 <0x0A> 1000 | 3000 <0x0A> 1000 | 3000 <0x0A> 2000 | 3000 <0x0A> 2000 | 3000 <0x0A> 2000 | 3000 <0x0A> 3000 | 3000 <0x0A> 3000 | 3000 <0x0A> 3000 | 3000 <0x0A> 4000 | 3000 <0x0A> 4000 | 3000 <0x0A> 4000 | 3000 <0x0A> 4000 | 3000 <0x0A> 5000 | 2000 <0x0A> 5000 | 2000 <0x0A> 5000 | 2000 <0x0A> 6000 | 1000 <0x0A> 6000 | 1000 <0x0A> 6000 | 0`\n![Scatter plot](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5570735%2F65ef5cbd4480839455ff150e5bc0817d%2Fs.jpg?generation=1686064090717212&alt=media)", "2290175": "Thank you for your continued effort on this data, after these competition changes I would expect this to become even more valuable.", "2290505": "Thank you for your efforts!", "2290665": "Good job. @brendanartley How can these generated images be utilized? Can they be used as pre-trained data or be trained in conjunction with competition training data?", "2291372": "Either could be a good strategy.\n\nI'm not sure how much to oversample the extracted data though as the private LB (61% of the score) comes from a different source. [Data Splits](https://www.kaggle.com/competitions/benetech-making-graphs-accessible/data).", "2291438": "Have you measured how much of the data you generated contributes to the final performance of the model, for example 100k, 200k, 300k..?", "2291496": "I'm not sure I understand the question. Currently I train on all 500k images for 1 epoch.", "2291515": "One additional question: 500k images with 1 epoch => .59 is that correct?", "2291521": "If I would do it, I would go with the pretraining with generated + competition and then fine-tune on only competition data. @marbury", "2291547": "Yeah, that is correct.", "2293262": "May I ask the max_patches you are using?", "2293429": "Hi,[@brendanartley](https://www.kaggle.com/brendanartley),thank you for your data. When I use these additional data for training, loss is very unstable compared to using only the original data.Is this normal?", "2293812": "Max patches is set to 1024. Training loss is stable for me.", "2293929": "I mean suppose we use 1M generated images, will the performance of model be better?", "2294381": "Models need sufficient information to learn patterns. Once you start adding new images it will reach a point where adding more images won't add any variance. The model will have learned all the patterns it can. What I mean it's not only about the number of images you train with but also the variance in data. What you can expect is that if you add some couple thousands images you will see a boost in performance but later adding more images will just produce a very little boost. This relates well with the law of diminishing marginal returns. @marbury", "2294847": "hi what's r2_score meaning in nr function", "2295014": "The `nr()` function is just shorthand for `normalized_rmse()`. I used functions which come from the [Competition Metric - Benetech Mixed-Match Notebook](https://www.kaggle.com/code/ryanholbrook/competition-metric-benetech-mixed-match).", "2295498": "thx so much" }, "source": "meta" }