{
  "id": 67132,
  "title": "Submission error, number of rows doesn't match?",
  "url": "/competitions/ga-customer-revenue-prediction/discussion/67132",
  "author_name": "",
  "post_date": "2018-09-29T07:27:10.913264Z",
  "votes": 1,
  "comment_count": 3,
  "views": 0,
  "content": "<p>I used submission code from public kernels as follows:</p>\n\n<pre><code>sub_df = pd.DataFrame({\"fullVisitorId\":test_id})\npred_test[pred_test&amp;lt;0] = 0\nsub_df[\"PredictedLogRevenue\"] = np.expm1(pred_test)\nsub_df = sub_df.groupby(\"fullVisitorId\")[\"PredictedLogRevenue\"].sum().reset_index()\nsub_df.columns = [\"fullVisitorId\", \"PredictedLogRevenue\"]\nsub_df[\"PredictedLogRevenue\"] = np.log1p(sub_df[\"PredictedLogRevenue\"])\nsub_df.to_csv(\"baseline_lgb.csv\", index=False)\n</code></pre>\n\n<p>But I notice the submission file have 650489 rows while the instruction said there is 617242 prediction rows, and I back to <code>len(test_df['fullVisitorId'].unique())</code> and that gives alos 650489 unique fullVisitorId, I don't understand what went wrong, please help, Thanks!</p>\n\n<p>Here is the submission error:</p>\n\n<pre><code>ERROR: The value '1000035437777728789' in the key column 'fullVisitorId' has already been defined (Line 475703, Column 1)\n\nERROR: The value '1000411008793922262' in the key column 'fullVisitorId' has already been defined (Line 475709, Column 1)\n\nERROR: The value '1000803807207540047' in the key column 'fullVisitorId' has already been defined (Line 475715, Column 1)\n\nERROR: The value '1000829672312727061' in the key column 'fullVisitorId' has already been defined (Line 475716, Column 1)\n\nERROR: The value '100104898171048438' in the key column 'fullVisitorId' has already been defined (Line 475723, Column 1)\n\nERROR: The value '100135695020255590' in the key column 'fullVisitorId' has already been defined (Line 475729, Column 1)\n\nERROR: The value '1001412656277263218' in the key column 'fullVisitorId' has already been defined (Line 475730, Column 1)\n\nERROR: The value '1002079863795050830' in the key column 'fullVisitorId' has already been defined (Line 475738, Column 1)\n\nERROR: The value '1002099307474679812' in the key column 'fullVisitorId' has already been defined (Line 475740, Column 1)\n\nERROR: The value '1002145056479546512' in the key column 'fullVisitorId' has already been defined (Line 475742, Column 1)\n</code></pre>",
  "messages": [
    {
      "id": "395704",
      "postDate": "09/29/2018 07:27:10",
      "content": "<p>I used submission code from public kernels as follows:</p>\n\n<pre><code>sub_df = pd.DataFrame({\"fullVisitorId\":test_id})\npred_test[pred_test&amp;lt;0] = 0\nsub_df[\"PredictedLogRevenue\"] = np.expm1(pred_test)\nsub_df = sub_df.groupby(\"fullVisitorId\")[\"PredictedLogRevenue\"].sum().reset_index()\nsub_df.columns = [\"fullVisitorId\", \"PredictedLogRevenue\"]\nsub_df[\"PredictedLogRevenue\"] = np.log1p(sub_df[\"PredictedLogRevenue\"])\nsub_df.to_csv(\"baseline_lgb.csv\", index=False)\n</code></pre>\n\n<p>But I notice the submission file have 650489 rows while the instruction said there is 617242 prediction rows, and I back to <code>len(test_df['fullVisitorId'].unique())</code> and that gives alos 650489 unique fullVisitorId, I don't understand what went wrong, please help, Thanks!</p>\n\n<p>Here is the submission error:</p>\n\n<pre><code>ERROR: The value '1000035437777728789' in the key column 'fullVisitorId' has already been defined (Line 475703, Column 1)\n\nERROR: The value '1000411008793922262' in the key column 'fullVisitorId' has already been defined (Line 475709, Column 1)\n\nERROR: The value '1000803807207540047' in the key column 'fullVisitorId' has already been defined (Line 475715, Column 1)\n\nERROR: The value '1000829672312727061' in the key column 'fullVisitorId' has already been defined (Line 475716, Column 1)\n\nERROR: The value '100104898171048438' in the key column 'fullVisitorId' has already been defined (Line 475723, Column 1)\n\nERROR: The value '100135695020255590' in the key column 'fullVisitorId' has already been defined (Line 475729, Column 1)\n\nERROR: The value '1001412656277263218' in the key column 'fullVisitorId' has already been defined (Line 475730, Column 1)\n\nERROR: The value '1002079863795050830' in the key column 'fullVisitorId' has already been defined (Line 475738, Column 1)\n\nERROR: The value '1002099307474679812' in the key column 'fullVisitorId' has already been defined (Line 475740, Column 1)\n\nERROR: The value '1002145056479546512' in the key column 'fullVisitorId' has already been defined (Line 475742, Column 1)\n</code></pre>",
      "rawMarkdown": "I used submission code from public kernels as follows:\n\n    sub_df = pd.DataFrame({\"fullVisitorId\":test_id})\n    pred_test[pred_test&lt;0] = 0\n    sub_df[\"PredictedLogRevenue\"] = np.expm1(pred_test)\n    sub_df = sub_df.groupby(\"fullVisitorId\")[\"PredictedLogRevenue\"].sum().reset_index()\n    sub_df.columns = [\"fullVisitorId\", \"PredictedLogRevenue\"]\n    sub_df[\"PredictedLogRevenue\"] = np.log1p(sub_df[\"PredictedLogRevenue\"])\n    sub_df.to_csv(\"baseline_lgb.csv\", index=False)\n\nBut I notice the submission file have 650489 rows while the instruction said there is 617242 prediction rows, and I back to `len(test_df['fullVisitorId'].unique())` and that gives alos 650489 unique fullVisitorId, I don't understand what went wrong, please help, Thanks!\n\nHere is the submission error:\n\n    ERROR: The value '1000035437777728789' in the key column 'fullVisitorId' has already been defined (Line 475703, Column 1)\n\n    ERROR: The value '1000411008793922262' in the key column 'fullVisitorId' has already been defined (Line 475709, Column 1)\n    \n    ERROR: The value '1000803807207540047' in the key column 'fullVisitorId' has already been defined (Line 475715, Column 1)\n    \n    ERROR: The value '1000829672312727061' in the key column 'fullVisitorId' has already been defined (Line 475716, Column 1)\n    \n    ERROR: The value '100104898171048438' in the key column 'fullVisitorId' has already been defined (Line 475723, Column 1)\n    \n    ERROR: The value '100135695020255590' in the key column 'fullVisitorId' has already been defined (Line 475729, Column 1)\n    \n    ERROR: The value '1001412656277263218' in the key column 'fullVisitorId' has already been defined (Line 475730, Column 1)\n    \n    ERROR: The value '1002079863795050830' in the key column 'fullVisitorId' has already been defined (Line 475738, Column 1)\n    \n    ERROR: The value '1002099307474679812' in the key column 'fullVisitorId' has already been defined (Line 475740, Column 1)\n    \n    ERROR: The value '1002145056479546512' in the key column 'fullVisitorId' has already been defined (Line 475742, Column 1)",
      "votes": null
    },
    {
      "id": "395713",
      "postDate": "09/29/2018 07:49:36",
      "content": "<p>hi ,\nI met the some error, but sometimes the answer is correct. do you have the solution?</p>",
      "rawMarkdown": "hi ,\nI met the some error, but sometimes the answer is correct. do you have the solution?",
      "votes": null
    },
    {
      "id": "395742",
      "postDate": "09/29/2018 08:50:52",
      "content": "<p>I think this error happens when you do not read the <code>fullVisitorId</code> column as string. Use the <code>dtype={'fullVisitorId': np.str}</code> in <code>pd.read_csv()</code> to fix this.</p>",
      "rawMarkdown": "I think this error happens when you do not read the `fullVisitorId` column as string. Use the `dtype={'fullVisitorId': np.str}` in `pd.read_csv()` to fix this.",
      "votes": null
    },
    {
      "id": "396098",
      "postDate": "09/30/2018 02:37:53",
      "content": "<p>Thanks! It worked!</p>",
      "rawMarkdown": "Thanks! It worked!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 395713,
      "author_name": "kangzhang",
      "author_url": "",
      "post_date": "09/29/2018 07:49:36",
      "content": "<p>hi ,\nI met the some error, but sometimes the answer is correct. do you have the solution?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 395742,
      "author_name": "mithrillion",
      "author_url": "",
      "post_date": "09/29/2018 08:50:52",
      "content": "<p>I think this error happens when you do not read the <code>fullVisitorId</code> column as string. Use the <code>dtype={'fullVisitorId': np.str}</code> in <code>pd.read_csv()</code> to fix this.</p>",
      "votes": null,
      "replies": [
        {
          "id": 396098,
          "author_name": "dewandforest",
          "author_url": "",
          "post_date": "09/30/2018 02:37:53",
          "content": "<p>Thanks! It worked!</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "395704": "I used submission code from public kernels as follows:\n\n    sub_df = pd.DataFrame({\"fullVisitorId\":test_id})\n    pred_test[pred_test&lt;0] = 0\n    sub_df[\"PredictedLogRevenue\"] = np.expm1(pred_test)\n    sub_df = sub_df.groupby(\"fullVisitorId\")[\"PredictedLogRevenue\"].sum().reset_index()\n    sub_df.columns = [\"fullVisitorId\", \"PredictedLogRevenue\"]\n    sub_df[\"PredictedLogRevenue\"] = np.log1p(sub_df[\"PredictedLogRevenue\"])\n    sub_df.to_csv(\"baseline_lgb.csv\", index=False)\n\nBut I notice the submission file have 650489 rows while the instruction said there is 617242 prediction rows, and I back to `len(test_df['fullVisitorId'].unique())` and that gives alos 650489 unique fullVisitorId, I don't understand what went wrong, please help, Thanks!\n\nHere is the submission error:\n\n    ERROR: The value '1000035437777728789' in the key column 'fullVisitorId' has already been defined (Line 475703, Column 1)\n\n    ERROR: The value '1000411008793922262' in the key column 'fullVisitorId' has already been defined (Line 475709, Column 1)\n    \n    ERROR: The value '1000803807207540047' in the key column 'fullVisitorId' has already been defined (Line 475715, Column 1)\n    \n    ERROR: The value '1000829672312727061' in the key column 'fullVisitorId' has already been defined (Line 475716, Column 1)\n    \n    ERROR: The value '100104898171048438' in the key column 'fullVisitorId' has already been defined (Line 475723, Column 1)\n    \n    ERROR: The value '100135695020255590' in the key column 'fullVisitorId' has already been defined (Line 475729, Column 1)\n    \n    ERROR: The value '1001412656277263218' in the key column 'fullVisitorId' has already been defined (Line 475730, Column 1)\n    \n    ERROR: The value '1002079863795050830' in the key column 'fullVisitorId' has already been defined (Line 475738, Column 1)\n    \n    ERROR: The value '1002099307474679812' in the key column 'fullVisitorId' has already been defined (Line 475740, Column 1)\n    \n    ERROR: The value '1002145056479546512' in the key column 'fullVisitorId' has already been defined (Line 475742, Column 1)",
    "395713": "hi ,\nI met the some error, but sometimes the answer is correct. do you have the solution?",
    "395742": "I think this error happens when you do not read the `fullVisitorId` column as string. Use the `dtype={'fullVisitorId': np.str}` in `pd.read_csv()` to fix this.",
    "396098": "Thanks! It worked!"
  },
  "source": "meta"
}