{
  "id": 667046,
  "title": "Reverse engineering the sine formula with high numerical precision",
  "content": "<p>In this competition, I learnt a lot about the importance of exploratory data analysis and high precision in numerical computing.</p>\n<p>Note: I did not use a multi-layer perceptron to generate my test set <code>y</code> label submission.</p>\n<p>Let's walk through how I built my model from the beginning.</p>\n<p>At the beginning, I used <a href=\"https://auto.gluon.ai/stable/index.html\" target=\"_blank\">AutoGluon</a> (an automatic machine learning library) to build a baseline model. I used <code>medium_quality</code>, which trains several tree-based models (LightGBM, XGBoost, and Random Forest) and neural networks and takes a weighted average of the predictions as the final result for better performance compared to one model. Note that the model choice is not important; you can train an MLP or Gradient Boosting Regressor as well.</p>\n<p>I submitted to the leaderboard and I got <code>0.03318</code>. That was quite good, but clearly not the highest possible score I could get as evidenced by the leaderboard ranking at that time. Given that the leaderboard's highest score at that time was around <code>0.003</code>, a 10% error compared to my submission, I guessed something interesting might lie inside the data.</p>\n<p>So I quickly ran some data visualizations, and here is what I got:</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F29566806%2Fdf7cd62be6450369a9c593f288fa5807%2Fcontour.png?generation=1768072409620546&amp;alt=media\" alt=\"\"></p>\n<p>Hmm, the dataset definitely looks synthetically generated. And there is some sort of symmetry going on. Without a clue, I asked Gemini 3.0 Pro if we could hand-craft a formula by reparameterizing based on x1 and x2. It turns out that by plotting <code>y</code> against \\(r = \\sqrt{x_1^2 + x_2^2}\\), I got:</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F29566806%2F547d26a6e02372a7c3371dab0dca9648%2Fradius-transformation.png?generation=1768072951525915&amp;alt=media\" alt=\"\"></p>\n<p>That looked a lot clearer. I was quite certain that the dataset had something to do with sine curves. And it looked like the sine curves were generated with an <em>increasing frequency</em>.</p>\n<p>That was an interesting property. After brainstorming, I came up with the following formulation of the equation:</p>\n<p>$$y = A \\cdot \\sin(B \\cdot r^2 + C \\cdot r + D) + E$$</p>\n<p>Where:</p>\n<ul>\n<li>\\(A\\): Amplitude</li>\n<li>\\(B, C\\): Frequency and frequency-rate</li>\n<li>\\(D\\): Phase</li>\n<li>\\(E\\): Vertical offset</li>\n</ul>\n<p>Let's throw it into <code>scipy</code> and fit the formula:</p>\n<pre><code>import matplotlib.pyplot as plt\nimport numpy as np\nimport pandas as pd\nfrom scipy.optimize import curve_fit\nfrom sklearn.metrics import mean_squared_error\nfrom sklearn.model_selection import KFold\n\ntrain_df = pd.read_csv(\"train.csv\")\ntest_df = pd.read_csv(\"test.csv\")\n\ntrain_df[\"r\"] = np.sqrt(train_df[\"x1\"] ** 2 + train_df[\"x2\"] ** 2)\ntest_df[\"r\"] = np.sqrt(test_df[\"x1\"] ** 2 + test_df[\"x2\"] ** 2)\n\nX_r = train_df[\"r\"].values\ny = train_df[\"y\"].values\n\n\ndef model(r, A, B, C, D, E):\n    return A * np.sin(B * r**2 + C * r + D) + E\n\n\np0 = [0.5, 5.0, 2.0, 0.0, 1.0]\nkf = KFold(n_splits=5, shuffle=True, random_state=42)\n\ncv_mses = []\nfor train_idx, val_idx in kf.split(train_df):\n    X_tr, y_tr = X_r[train_idx], y[train_idx]\n    X_va, y_va = X_r[val_idx], y[val_idx]\n\n    popt, _ = curve_fit(model, X_tr, y_tr, p0=p0)\n\n    y_pred = model(X_va, *popt)\n    cv_mses.append(mean_squared_error(y_va, y_pred))\n\nprint(f\"Model CV MSE: {np.mean(cv_mses):.6f} +/- {np.std(cv_mses):.6f}\")\n\npopt_final, _ = curve_fit(model, X_r, y, p0=p0)\nprint(f\"Final Parameters: {popt_final}\")\n\nr_grid = np.linspace(0, 1.5, 1000)\ny_fit = model(r_grid, *popt_final)\nplt.figure(figsize=(12, 6))\nplt.scatter(X_r, y, s=1, alpha=0.1, color=\"blue\", label=\"Training Data\")\nplt.plot(r_grid, y_fit, \"r-\", lw=3, label=\"Curve Fit\")\nplt.xlabel(\"Radius r\")\nplt.ylabel(\"y\")\nplt.title(f\"Global Curve Fit (CV MSE: {np.mean(cv_mses):.6f})\")\nplt.legend()\nplt.show()\n\ny_test_pred = model(test_df[\"r\"].values, *popt_final)\nsubmission_df = pd.DataFrame({\"id\": np.arange(1, 2001), \"y\": y_test_pred})\nsubmission_df.to_csv(\"submission.csv\", index=False)\n</code></pre>\n<p>This gave me</p>\n<p>$$ y = 0.5008 \\sin(6.0419 r^2 - 0.0602 r + 0.0175) + 1.0001 $$</p>\n<pre><code>Model CV MSE: 0.022197 +/- 0.000680\nFinal Parameters: [ 0.50083101  6.04194113 -0.06023903  0.01753956  1.00005388]\n</code></pre>\n<p>I submitted to the leaderboard, and it gave me <code>0.00300</code>.</p>\n<p>While the results are great, the decimal places of the variables look a little bit off. Given that I assumed the dataset was likely synthetically generated, I made a few <a href=\"https://en.wikipedia.org/wiki/Inductive_bias\" target=\"_blank\">inductive biases</a>:</p>\n<ul>\n<li>The dataset is hand-crafted by a human</li>\n<li>The true formula responsible for generating the samples must be simple</li>\n<li>For the values in the formula, prefer integers over floating point numbers</li>\n</ul>\n<p>That way, I simplified the formula into:</p>\n<p>$$ y = 0.5 \\sin(6 r^2) + 1 $$</p>\n<p>This gave me a leaderboard score of <code>0.00247</code>.</p>\n<p>Now, I notice that I'm still not the highest in the leaderboard ranking. This could be due to several reasons, for example, <a href=\"https://www.kaggle.com/docs/competitions#leaderboard:~:text=In%20the%20event%20of%20an%20exact%20score%20tie%2C%20the%20tiebreaker%20is%20the%20team%20which%20submitted%20earlier.%20Kaggle%20always%20uses%20full%20precision%20when%20determining%20rankings%2C%20not%20just%20the%20truncated%20precision%20shown%20on%20the%20Leaderboard.\" target=\"_blank\">Kaggle ranks the first person to submit higher if 2 users have the same score. However, Kaggle also evaluates the leaderboard metric with <em>full precision!</em></a></p>\n<p>I looked into the <code>submission.csv</code> I had generated:</p>\n<pre><code>id,y\n1,0.5015993834619267\n2,1.467676273657998\n3,0.6799769398701817\n4,1.3485892841618887\n5,1.4317098100334664\n6,1.4830281438178594\n</code></pre>\n<p>Given that the formula was a sine curve, which computers can only approximate instead of precisely calculating, I guessed the 16-decimal-place precision in the generated CSV was not enough.</p>\n<p>From what I've learnt in my Computer Organization class, we can use higher-precision data types to calculate the equation more accurately.</p>\n<p>So I came up with code that uses the <code>mpmath</code> library to calculate the equation with a precision of <code>1000</code> decimal places.</p>\n<pre><code>import pandas as pd\nimport numpy as np\nimport mpmath as mp\n\nmp.mp.dps = 1000\n\ntest_df = pd.read_csv(\"test.csv\", dtype={\"x1\": \"string\", \"x2\": \"string\"})\ny_str = []\nfor s1, s2 in zip(test_df[\"x1\"], test_df[\"x2\"]):\n    x1 = mp.mpf(s1)\n    x2 = mp.mpf(s2)\n    r2 = x1 * x1 + x2 * x2\n    y = mp.mpf(\"1.0\") + mp.mpf(\"0.5\") * mp.sin(mp.mpf(\"6.0\") * r2)\n    y_str.append(mp.nstr(y, n=1000))\n\nsubmission_df = pd.DataFrame({\"id\": np.arange(1, len(y_str) + 1), \"y\": y_str})\nsubmission_df.to_csv(\"submission.highprec.csv\", index=False)\n</code></pre>\n<pre><code>id,y\n1,0.5020010035360700362206597987786111594123629203358911339986548017850158905600859624002760847100101825697287740091135184314121656099284776859060497464055326926497526579695155903427397807547335046591328437414112053491467098697123170383443093995673538365205906606598644579196408660325326207702152674767992697337469445205249467267461159328532940235671828697134392104897840619129686044876977490052755418447038765050002790158456973136828515036168950085646898585581163114283837881893419888916048993719924760483147981640195213664576822883955370330805606427682016319769403913699298048954607869454199696624623522313392965222617573250637046702242516828514221318261959544818487150798003601533338542222064801402146597805840478120953298110953341944499988669822935832411005456448468153095677237213760359000128768411674740636908858852447189469698118394833456613496772417505835189486470543416150138239264288877096831448733942264472455617271948744732050852303308487799689916152250772489866587741914356802089109051589033\n</code></pre>\n<p>This looked good! I then submitted this CSV as my final submission on the leaderboard. My leaderboard ranking went slightly higher. So I declared this a success and stopped working on it.</p>\n<h1>So, who are you?</h1>\n<p>I am a random undergraduate student from the University of Hong Kong, currently on winter break. I was looking for fun Kaggle competitions. Thank you to the competition host for providing such an interesting dataset!</p>",
  "comments": [],
  "comment_count": 0,
  "messages": [
    {
      "id": 3389281,
      "postDate": "2026-01-10T19:10:22.763Z",
      "content": "<p>In this competition, I learnt a lot about the importance of exploratory data analysis and high precision in numerical computing.</p>\n<p>Note: I did not use a multi-layer perceptron to generate my test set <code>y</code> label submission.</p>\n<p>Let's walk through how I built my model from the beginning.</p>\n<p>At the beginning, I used <a href=\"https://auto.gluon.ai/stable/index.html\" target=\"_blank\">AutoGluon</a> (an automatic machine learning library) to build a baseline model. I used <code>medium_quality</code>, which trains several tree-based models (LightGBM, XGBoost, and Random Forest) and neural networks and takes a weighted average of the predictions as the final result for better performance compared to one model. Note that the model choice is not important; you can train an MLP or Gradient Boosting Regressor as well.</p>\n<p>I submitted to the leaderboard and I got <code>0.03318</code>. That was quite good, but clearly not the highest possible score I could get as evidenced by the leaderboard ranking at that time. Given that the leaderboard's highest score at that time was around <code>0.003</code>, a 10% error compared to my submission, I guessed something interesting might lie inside the data.</p>\n<p>So I quickly ran some data visualizations, and here is what I got:</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F29566806%2Fdf7cd62be6450369a9c593f288fa5807%2Fcontour.png?generation=1768072409620546&amp;alt=media\" alt=\"\"></p>\n<p>Hmm, the dataset definitely looks synthetically generated. And there is some sort of symmetry going on. Without a clue, I asked Gemini 3.0 Pro if we could hand-craft a formula by reparameterizing based on x1 and x2. It turns out that by plotting <code>y</code> against \\(r = \\sqrt{x_1^2 + x_2^2}\\), I got:</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F29566806%2F547d26a6e02372a7c3371dab0dca9648%2Fradius-transformation.png?generation=1768072951525915&amp;alt=media\" alt=\"\"></p>\n<p>That looked a lot clearer. I was quite certain that the dataset had something to do with sine curves. And it looked like the sine curves were generated with an <em>increasing frequency</em>.</p>\n<p>That was an interesting property. After brainstorming, I came up with the following formulation of the equation:</p>\n<p>$$y = A \\cdot \\sin(B \\cdot r^2 + C \\cdot r + D) + E$$</p>\n<p>Where:</p>\n<ul>\n<li>\\(A\\): Amplitude</li>\n<li>\\(B, C\\): Frequency and frequency-rate</li>\n<li>\\(D\\): Phase</li>\n<li>\\(E\\): Vertical offset</li>\n</ul>\n<p>Let's throw it into <code>scipy</code> and fit the formula:</p>\n<pre><code>import matplotlib.pyplot as plt\nimport numpy as np\nimport pandas as pd\nfrom scipy.optimize import curve_fit\nfrom sklearn.metrics import mean_squared_error\nfrom sklearn.model_selection import KFold\n\ntrain_df = pd.read_csv(\"train.csv\")\ntest_df = pd.read_csv(\"test.csv\")\n\ntrain_df[\"r\"] = np.sqrt(train_df[\"x1\"] ** 2 + train_df[\"x2\"] ** 2)\ntest_df[\"r\"] = np.sqrt(test_df[\"x1\"] ** 2 + test_df[\"x2\"] ** 2)\n\nX_r = train_df[\"r\"].values\ny = train_df[\"y\"].values\n\n\ndef model(r, A, B, C, D, E):\n    return A * np.sin(B * r**2 + C * r + D) + E\n\n\np0 = [0.5, 5.0, 2.0, 0.0, 1.0]\nkf = KFold(n_splits=5, shuffle=True, random_state=42)\n\ncv_mses = []\nfor train_idx, val_idx in kf.split(train_df):\n    X_tr, y_tr = X_r[train_idx], y[train_idx]\n    X_va, y_va = X_r[val_idx], y[val_idx]\n\n    popt, _ = curve_fit(model, X_tr, y_tr, p0=p0)\n\n    y_pred = model(X_va, *popt)\n    cv_mses.append(mean_squared_error(y_va, y_pred))\n\nprint(f\"Model CV MSE: {np.mean(cv_mses):.6f} +/- {np.std(cv_mses):.6f}\")\n\npopt_final, _ = curve_fit(model, X_r, y, p0=p0)\nprint(f\"Final Parameters: {popt_final}\")\n\nr_grid = np.linspace(0, 1.5, 1000)\ny_fit = model(r_grid, *popt_final)\nplt.figure(figsize=(12, 6))\nplt.scatter(X_r, y, s=1, alpha=0.1, color=\"blue\", label=\"Training Data\")\nplt.plot(r_grid, y_fit, \"r-\", lw=3, label=\"Curve Fit\")\nplt.xlabel(\"Radius r\")\nplt.ylabel(\"y\")\nplt.title(f\"Global Curve Fit (CV MSE: {np.mean(cv_mses):.6f})\")\nplt.legend()\nplt.show()\n\ny_test_pred = model(test_df[\"r\"].values, *popt_final)\nsubmission_df = pd.DataFrame({\"id\": np.arange(1, 2001), \"y\": y_test_pred})\nsubmission_df.to_csv(\"submission.csv\", index=False)\n</code></pre>\n<p>This gave me</p>\n<p>$$ y = 0.5008 \\sin(6.0419 r^2 - 0.0602 r + 0.0175) + 1.0001 $$</p>\n<pre><code>Model CV MSE: 0.022197 +/- 0.000680\nFinal Parameters: [ 0.50083101  6.04194113 -0.06023903  0.01753956  1.00005388]\n</code></pre>\n<p>I submitted to the leaderboard, and it gave me <code>0.00300</code>.</p>\n<p>While the results are great, the decimal places of the variables look a little bit off. Given that I assumed the dataset was likely synthetically generated, I made a few <a href=\"https://en.wikipedia.org/wiki/Inductive_bias\" target=\"_blank\">inductive biases</a>:</p>\n<ul>\n<li>The dataset is hand-crafted by a human</li>\n<li>The true formula responsible for generating the samples must be simple</li>\n<li>For the values in the formula, prefer integers over floating point numbers</li>\n</ul>\n<p>That way, I simplified the formula into:</p>\n<p>$$ y = 0.5 \\sin(6 r^2) + 1 $$</p>\n<p>This gave me a leaderboard score of <code>0.00247</code>.</p>\n<p>Now, I notice that I'm still not the highest in the leaderboard ranking. This could be due to several reasons, for example, <a href=\"https://www.kaggle.com/docs/competitions#leaderboard:~:text=In%20the%20event%20of%20an%20exact%20score%20tie%2C%20the%20tiebreaker%20is%20the%20team%20which%20submitted%20earlier.%20Kaggle%20always%20uses%20full%20precision%20when%20determining%20rankings%2C%20not%20just%20the%20truncated%20precision%20shown%20on%20the%20Leaderboard.\" target=\"_blank\">Kaggle ranks the first person to submit higher if 2 users have the same score. However, Kaggle also evaluates the leaderboard metric with <em>full precision!</em></a></p>\n<p>I looked into the <code>submission.csv</code> I had generated:</p>\n<pre><code>id,y\n1,0.5015993834619267\n2,1.467676273657998\n3,0.6799769398701817\n4,1.3485892841618887\n5,1.4317098100334664\n6,1.4830281438178594\n</code></pre>\n<p>Given that the formula was a sine curve, which computers can only approximate instead of precisely calculating, I guessed the 16-decimal-place precision in the generated CSV was not enough.</p>\n<p>From what I've learnt in my Computer Organization class, we can use higher-precision data types to calculate the equation more accurately.</p>\n<p>So I came up with code that uses the <code>mpmath</code> library to calculate the equation with a precision of <code>1000</code> decimal places.</p>\n<pre><code>import pandas as pd\nimport numpy as np\nimport mpmath as mp\n\nmp.mp.dps = 1000\n\ntest_df = pd.read_csv(\"test.csv\", dtype={\"x1\": \"string\", \"x2\": \"string\"})\ny_str = []\nfor s1, s2 in zip(test_df[\"x1\"], test_df[\"x2\"]):\n    x1 = mp.mpf(s1)\n    x2 = mp.mpf(s2)\n    r2 = x1 * x1 + x2 * x2\n    y = mp.mpf(\"1.0\") + mp.mpf(\"0.5\") * mp.sin(mp.mpf(\"6.0\") * r2)\n    y_str.append(mp.nstr(y, n=1000))\n\nsubmission_df = pd.DataFrame({\"id\": np.arange(1, len(y_str) + 1), \"y\": y_str})\nsubmission_df.to_csv(\"submission.highprec.csv\", index=False)\n</code></pre>\n<pre><code>id,y\n1,0.5020010035360700362206597987786111594123629203358911339986548017850158905600859624002760847100101825697287740091135184314121656099284776859060497464055326926497526579695155903427397807547335046591328437414112053491467098697123170383443093995673538365205906606598644579196408660325326207702152674767992697337469445205249467267461159328532940235671828697134392104897840619129686044876977490052755418447038765050002790158456973136828515036168950085646898585581163114283837881893419888916048993719924760483147981640195213664576822883955370330805606427682016319769403913699298048954607869454199696624623522313392965222617573250637046702242516828514221318261959544818487150798003601533338542222064801402146597805840478120953298110953341944499988669822935832411005456448468153095677237213760359000128768411674740636908858852447189469698118394833456613496772417505835189486470543416150138239264288877096831448733942264472455617271948744732050852303308487799689916152250772489866587741914356802089109051589033\n</code></pre>\n<p>This looked good! I then submitted this CSV as my final submission on the leaderboard. My leaderboard ranking went slightly higher. So I declared this a success and stopped working on it.</p>\n<h1>So, who are you?</h1>\n<p>I am a random undergraduate student from the University of Hong Kong, currently on winter break. I was looking for fun Kaggle competitions. Thank you to the competition host for providing such an interesting dataset!</p>",
      "rawMarkdown": "In this competition, I learnt a lot about the importance of exploratory data analysis and high precision in numerical computing.\n\nNote: I did not use a multi-layer perceptron to generate my test set `y` label submission.\n\nLet's walk through how I built my model from the beginning.\n\nAt the beginning, I used [AutoGluon](https://auto.gluon.ai/stable/index.html) (an automatic machine learning library) to build a baseline model. I used `medium_quality`, which trains several tree-based models (LightGBM, XGBoost, and Random Forest) and neural networks and takes a weighted average of the predictions as the final result for better performance compared to one model. Note that the model choice is not important; you can train an MLP or Gradient Boosting Regressor as well.\n\nI submitted to the leaderboard and I got `0.03318`. That was quite good, but clearly not the highest possible score I could get as evidenced by the leaderboard ranking at that time. Given that the leaderboard's highest score at that time was around `0.003`, a 10% error compared to my submission, I guessed something interesting might lie inside the data.\n\nSo I quickly ran some data visualizations, and here is what I got:\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F29566806%2Fdf7cd62be6450369a9c593f288fa5807%2Fcontour.png?generation=1768072409620546&alt=media)\n\nHmm, the dataset definitely looks synthetically generated. And there is some sort of symmetry going on. Without a clue, I asked Gemini 3.0 Pro if we could hand-craft a formula by reparameterizing based on x1 and x2. It turns out that by plotting `y` against \\\\(r = \\sqrt{x_1^2 + x_2^2}\\\\), I got:\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F29566806%2F547d26a6e02372a7c3371dab0dca9648%2Fradius-transformation.png?generation=1768072951525915&alt=media)\n\nThat looked a lot clearer. I was quite certain that the dataset had something to do with sine curves. And it looked like the sine curves were generated with an *increasing frequency*.\n\nThat was an interesting property. After brainstorming, I came up with the following formulation of the equation:\n\n$$y = A \\cdot \\sin(B \\cdot r^2 + C \\cdot r + D) + E$$\n\nWhere:\n*   \\\\(A\\\\): Amplitude\n*   \\\\(B, C\\\\): Frequency and frequency-rate\n*   \\\\(D\\\\): Phase\n*   \\\\(E\\\\): Vertical offset\n\nLet's throw it into `scipy` and fit the formula:\n\n```\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport pandas as pd\nfrom scipy.optimize import curve_fit\nfrom sklearn.metrics import mean_squared_error\nfrom sklearn.model_selection import KFold\n\ntrain_df = pd.read_csv(\"train.csv\")\ntest_df = pd.read_csv(\"test.csv\")\n\ntrain_df[\"r\"] = np.sqrt(train_df[\"x1\"] ** 2 + train_df[\"x2\"] ** 2)\ntest_df[\"r\"] = np.sqrt(test_df[\"x1\"] ** 2 + test_df[\"x2\"] ** 2)\n\nX_r = train_df[\"r\"].values\ny = train_df[\"y\"].values\n\n\ndef model(r, A, B, C, D, E):\n    return A * np.sin(B * r**2 + C * r + D) + E\n\n\np0 = [0.5, 5.0, 2.0, 0.0, 1.0]\nkf = KFold(n_splits=5, shuffle=True, random_state=42)\n\ncv_mses = []\nfor train_idx, val_idx in kf.split(train_df):\n    X_tr, y_tr = X_r[train_idx], y[train_idx]\n    X_va, y_va = X_r[val_idx], y[val_idx]\n\n    popt, _ = curve_fit(model, X_tr, y_tr, p0=p0)\n\n    y_pred = model(X_va, *popt)\n    cv_mses.append(mean_squared_error(y_va, y_pred))\n\nprint(f\"Model CV MSE: {np.mean(cv_mses):.6f} +/- {np.std(cv_mses):.6f}\")\n\npopt_final, _ = curve_fit(model, X_r, y, p0=p0)\nprint(f\"Final Parameters: {popt_final}\")\n\nr_grid = np.linspace(0, 1.5, 1000)\ny_fit = model(r_grid, *popt_final)\nplt.figure(figsize=(12, 6))\nplt.scatter(X_r, y, s=1, alpha=0.1, color=\"blue\", label=\"Training Data\")\nplt.plot(r_grid, y_fit, \"r-\", lw=3, label=\"Curve Fit\")\nplt.xlabel(\"Radius r\")\nplt.ylabel(\"y\")\nplt.title(f\"Global Curve Fit (CV MSE: {np.mean(cv_mses):.6f})\")\nplt.legend()\nplt.show()\n\ny_test_pred = model(test_df[\"r\"].values, *popt_final)\nsubmission_df = pd.DataFrame({\"id\": np.arange(1, 2001), \"y\": y_test_pred})\nsubmission_df.to_csv(\"submission.csv\", index=False)\n```\n\nThis gave me\n\n$$ y = 0.5008 \\sin(6.0419 r^2 - 0.0602 r + 0.0175) + 1.0001 $$\n\n```\nModel CV MSE: 0.022197 +/- 0.000680\nFinal Parameters: [ 0.50083101  6.04194113 -0.06023903  0.01753956  1.00005388]\n```\n\nI submitted to the leaderboard, and it gave me `0.00300`.\n\nWhile the results are great, the decimal places of the variables look a little bit off. Given that I assumed the dataset was likely synthetically generated, I made a few [inductive biases](https://en.wikipedia.org/wiki/Inductive_bias):\n\n* The dataset is hand-crafted by a human\n* The true formula responsible for generating the samples must be simple\n* For the values in the formula, prefer integers over floating point numbers\n\nThat way, I simplified the formula into:\n\n$$ y = 0.5 \\sin(6 r^2) + 1 $$\n\nThis gave me a leaderboard score of `0.00247`.\n\nNow, I notice that I'm still not the highest in the leaderboard ranking. This could be due to several reasons, for example, [Kaggle ranks the first person to submit higher if 2 users have the same score. However, Kaggle also evaluates the leaderboard metric with *full precision!*](https://www.kaggle.com/docs/competitions#leaderboard:~:text=In%20the%20event%20of%20an%20exact%20score%20tie%2C%20the%20tiebreaker%20is%20the%20team%20which%20submitted%20earlier.%20Kaggle%20always%20uses%20full%20precision%20when%20determining%20rankings%2C%20not%20just%20the%20truncated%20precision%20shown%20on%20the%20Leaderboard.)\n\nI looked into the `submission.csv` I had generated:\n\n```\nid,y\n1,0.5015993834619267\n2,1.467676273657998\n3,0.6799769398701817\n4,1.3485892841618887\n5,1.4317098100334664\n6,1.4830281438178594\n```\n\nGiven that the formula was a sine curve, which computers can only approximate instead of precisely calculating, I guessed the 16-decimal-place precision in the generated CSV was not enough.\n\nFrom what I've learnt in my Computer Organization class, we can use higher-precision data types to calculate the equation more accurately.\n\nSo I came up with code that uses the `mpmath` library to calculate the equation with a precision of `1000` decimal places.\n\n```\nimport pandas as pd\nimport numpy as np\nimport mpmath as mp\n\nmp.mp.dps = 1000\n\ntest_df = pd.read_csv(\"test.csv\", dtype={\"x1\": \"string\", \"x2\": \"string\"})\ny_str = []\nfor s1, s2 in zip(test_df[\"x1\"], test_df[\"x2\"]):\n    x1 = mp.mpf(s1)\n    x2 = mp.mpf(s2)\n    r2 = x1 * x1 + x2 * x2\n    y = mp.mpf(\"1.0\") + mp.mpf(\"0.5\") * mp.sin(mp.mpf(\"6.0\") * r2)\n    y_str.append(mp.nstr(y, n=1000))\n\nsubmission_df = pd.DataFrame({\"id\": np.arange(1, len(y_str) + 1), \"y\": y_str})\nsubmission_df.to_csv(\"submission.highprec.csv\", index=False)\n```\n\n```\nid,y\n1,0.5020010035360700362206597987786111594123629203358911339986548017850158905600859624002760847100101825697287740091135184314121656099284776859060497464055326926497526579695155903427397807547335046591328437414112053491467098697123170383443093995673538365205906606598644579196408660325326207702152674767992697337469445205249467267461159328532940235671828697134392104897840619129686044876977490052755418447038765050002790158456973136828515036168950085646898585581163114283837881893419888916048993719924760483147981640195213664576822883955370330805606427682016319769403913699298048954607869454199696624623522313392965222617573250637046702242516828514221318261959544818487150798003601533338542222064801402146597805840478120953298110953341944499988669822935832411005456448468153095677237213760359000128768411674740636908858852447189469698118394833456613496772417505835189486470543416150138239264288877096831448733942264472455617271948744732050852303308487799689916152250772489866587741914356802089109051589033\n```\n\nThis looked good! I then submitted this CSV as my final submission on the leaderboard. My leaderboard ranking went slightly higher. So I declared this a success and stopped working on it.\n\n# So, who are you?\n\nI am a random undergraduate student from the University of Hong Kong, currently on winter break. I was looking for fun Kaggle competitions. Thank you to the competition host for providing such an interesting dataset!"
    }
  ],
  "complete": true,
  "source": "live",
  "fetched_at": "2026-10-20T05:53:47Z",
  "competition_id": 125114,
  "competition_slug": "function-approximation-using-multilayer-perceptron",
  "team_id": 15055531,
  "solution_url": "/competitions/function-approximation-using-multilayer-perceptron/writeups/reverse-engineering-the-sine-formula-with-high-num"
}