{
  "id": 66425,
  "title": "On The Use of External Data",
  "url": "/competitions/ga-customer-revenue-prediction/discussion/66425",
  "author_name": "Allen",
  "post_date": "2018-09-21T12:28:20.399000",
  "votes": 8,
  "comment_count": 14,
  "views": 0,
  "content": "<p>The rules from the competition state the following:</p>\n\n<blockquote>\n  <p>C. External Data. Unless otherwise expressly stated on the Competition Website, you may not use data other than the Competition Data to develop and test your models and Submissions. The Competition Sponsor reserves the right to disqualify any participant who Competition Sponsor discovers has used or attempted to use data other than the Competition Data in connection with the Competition, or who uses the Competition Data other than as permitted by the Competition Website and these Rules.</p>\n</blockquote>\n\n<p>How strictly is this interpreted? For instance, we are given geographic data. If I replace the country values with something like the Index of Economic Freedom (which is an external but open data source), would that be considered in violation of the rules?</p>",
  "messages": [
    {
      "id": 391222,
      "postDate": "2018-09-21T12:28:20.400Z",
      "content": "<p>The rules from the competition state the following:</p>\n\n<blockquote>\n  <p>C. External Data. Unless otherwise expressly stated on the Competition Website, you may not use data other than the Competition Data to develop and test your models and Submissions. The Competition Sponsor reserves the right to disqualify any participant who Competition Sponsor discovers has used or attempted to use data other than the Competition Data in connection with the Competition, or who uses the Competition Data other than as permitted by the Competition Website and these Rules.</p>\n</blockquote>\n\n<p>How strictly is this interpreted? For instance, we are given geographic data. If I replace the country values with something like the Index of Economic Freedom (which is an external but open data source), would that be considered in violation of the rules?</p>",
      "rawMarkdown": "The rules from the competition state the following:\n\n&gt; C. External Data. Unless otherwise expressly stated on the Competition Website, you may not use data other than the Competition Data to develop and test your models and Submissions. The Competition Sponsor reserves the right to disqualify any participant who Competition Sponsor discovers has used or attempted to use data other than the Competition Data in connection with the Competition, or who uses the Competition Data other than as permitted by the Competition Website and these Rules.\n\nHow strictly is this interpreted? For instance, we are given geographic data. If I replace the country values with something like the Index of Economic Freedom (which is an external but open data source), would that be considered in violation of the rules?",
      "votes": 8
    },
    {
      "id": 391240,
      "postDate": "2018-09-21T13:00:07.517Z",
      "content": "<p>Wondering the same but about time zone... Has anyone converted time to visitor's local time?</p>",
      "rawMarkdown": "Wondering the same but about time zone... Has anyone converted time to visitor's local time?",
      "votes": 4,
      "replies": [
        {
          "id": 391333,
          "postDate": "2018-09-21T16:09:26.763Z",
          "content": "<p>I don't think any type of time conversion would be considered a violation of the rules.</p>",
          "rawMarkdown": "I don't think any type of time conversion would be considered a violation of the rules.",
          "votes": 2
        },
        {
          "id": 391407,
          "postDate": "2018-09-21T18:39:27.087Z",
          "content": "<p>I’ve converted to local time without using external data - entirely possible.</p>",
          "rawMarkdown": "I’ve converted to local time without using external data - entirely possible."
        }
      ]
    },
    {
      "id": 393705,
      "postDate": "2018-09-25T19:24:26.293Z",
      "content": "<p>We have agreed with the host to allow the use of <strong>publicly, freely available data</strong>, as long as the source of any external data is posted to the official competition forum prior to the First Submission Deadline. I will open a new thread shortly for official sharing of those external data sources.</p>",
      "rawMarkdown": "We have agreed with the host to allow the use of **publicly, freely available data**, as long as the source of any external data is posted to the official competition forum prior to the First Submission Deadline. I will open a new thread shortly for official sharing of those external data sources.",
      "votes": 4,
      "replies": [
        {
          "id": 393768,
          "postDate": "2018-09-25T21:20:34.317Z",
          "content": "<p>Thank you!</p>",
          "rawMarkdown": "Thank you!"
        }
      ]
    },
    {
      "id": 393229,
      "postDate": "2018-09-25T02:29:06Z",
      "content": "<p>Thinking outside the box is one of the best things data scientists do when solving a problem. Many times a breakthrough comes from using additional data that no one previously considered. Transfer learning is pretty much built on this principle, as is knowledge sharing in general.</p>\n\n<p>I would find it helpful if someone could explain the rationale in preventing the use of \"external\" data.</p>",
      "rawMarkdown": "Thinking outside the box is one of the best things data scientists do when solving a problem. Many times a breakthrough comes from using additional data that no one previously considered. Transfer learning is pretty much built on this principle, as is knowledge sharing in general.\n\nI would find it helpful if someone could explain the rationale in preventing the use of \"external\" data.",
      "votes": 4,
      "replies": [
        {
          "id": 393478,
          "postDate": "2018-09-25T12:58:11.817Z",
          "content": "<p>Imagine I work for a company that has proprietary data that is comparable to what is used in this competition. If I integrated that data into the training of my model, I would have a competitive advantage against others in this competition who don't have access to that same data. Many would argue that this is unfair, and I suppose that that is why the rule is in place.</p>\n\n<p>That said, I don't know why we shouldn't be allowed to use external data that is open source and available to everyone (e.g. census data). My only guess is that maybe it makes it harder for them to detect cheating.</p>",
          "rawMarkdown": "Imagine I work for a company that has proprietary data that is comparable to what is used in this competition. If I integrated that data into the training of my model, I would have a competitive advantage against others in this competition who don't have access to that same data. Many would argue that this is unfair, and I suppose that that is why the rule is in place.\n\nThat said, I don't know why we shouldn't be allowed to use external data that is open source and available to everyone (e.g. census data). My only guess is that maybe it makes it harder for them to detect cheating.",
          "votes": 1
        },
        {
          "id": 393504,
          "postDate": "2018-09-25T13:38:37.307Z",
          "content": "<p>Makes sense with proprietary data giving an advantage.</p>",
          "rawMarkdown": "Makes sense with proprietary data giving an advantage."
        },
        {
          "id": 393526,
          "postDate": "2018-09-25T14:20:54.530Z",
          "content": "<p>So maybe <a href=\"/philculliton\">@philculliton</a> can help us here? Can we use publically available external data? </p>",
          "rawMarkdown": "So maybe @philculliton can help us here? Can we use publically available external data? "
        },
        {
          "id": 393543,
          "postDate": "2018-09-25T14:42:09.850Z",
          "content": "<p>Seeing the replies to Phil's last post, he may well be now taking a Sabbatical year on the North Pole..  Hope things quiet down with or without competition reset and we can move on.\nI would definitely allow external data as long as it is open source and is published on this forum.</p>",
          "rawMarkdown": "Seeing the replies to Phil's last post, he may well be now taking a Sabbatical year on the North Pole..  Hope things quiet down with or without competition reset and we can move on.\nI would definitely allow external data as long as it is open source and is published on this forum.",
          "votes": 3
        }
      ]
    },
    {
      "id": 420501,
      "postDate": "2018-11-13T18:20:57.763Z",
      "content": "<p>External data of other of any structured data can help your to provide the part where it having the part which user actually needs you can access the other devices via <a href=\"http://www.121techsupport.co/garmim-map-update/\">http://www.121techsupport.co/garmim-map-update/</a>  and this is going to provide your the entire parts of structured data. </p>",
      "rawMarkdown": "External data of other of any structured data can help your to provide the part where it having the part which user actually needs you can access the other devices via http://www.121techsupport.co/garmim-map-update/  and this is going to provide your the entire parts of structured data. ",
      "votes": -1
    },
    {
      "id": 392056,
      "postDate": "2018-09-22T23:06:09.453Z",
      "content": "<p>The external data where it will be going to have the particular fact which is going to provide the part where it is havi9ng the part that the user needs to raise the external device through the <a href=\"https://hpsupports.co/hp-support-australia/\">https://hpsupports.co/hp-support-australia/</a> which is going to provide the part all around it </p>",
      "rawMarkdown": "The external data where it will be going to have the particular fact which is going to provide the part where it is havi9ng the part that the user needs to raise the external device through the https://hpsupports.co/hp-support-australia/ which is going to provide the part all around it ",
      "votes": -4,
      "replies": [
        {
          "id": 393221,
          "postDate": "2018-09-25T02:13:17.873Z",
          "content": "<p>Hi Brat Pat. Why do you think Kagglers like to use external data? Also, how do you feel about Judg Kavanagh's prospects? </p>",
          "rawMarkdown": "Hi Brat Pat. Why do you think Kagglers like to use external data? Also, how do you feel about Judg Kavanagh's prospects? ",
          "votes": 1
        }
      ]
    },
    {
      "id": 393992,
      "postDate": "2018-09-26T07:02:31.067Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 391240,
      "author_name": "S D",
      "author_url": "",
      "post_date": "2018-09-21T13:00:07.517000",
      "content": "<p>Wondering the same but about time zone... Has anyone converted time to visitor's local time?</p>",
      "votes": 4,
      "replies": [
        {
          "id": 391333,
          "author_name": "Allen",
          "author_url": "",
          "post_date": "2018-09-21T16:09:26.763000",
          "content": "<p>I don't think any type of time conversion would be considered a violation of the rules.</p>",
          "votes": 2,
          "replies": []
        },
        {
          "id": 391407,
          "author_name": "DuraQ",
          "author_url": "",
          "post_date": "2018-09-21T18:39:27.087000",
          "content": "<p>I’ve converted to local time without using external data - entirely possible.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 393705,
      "author_name": "Julia Elliott",
      "author_url": "",
      "post_date": "2018-09-25T19:24:26.293000",
      "content": "<p>We have agreed with the host to allow the use of <strong>publicly, freely available data</strong>, as long as the source of any external data is posted to the official competition forum prior to the First Submission Deadline. I will open a new thread shortly for official sharing of those external data sources.</p>",
      "votes": 4,
      "replies": [
        {
          "id": 393768,
          "author_name": "Leonid Gilburd",
          "author_url": "",
          "post_date": "2018-09-25T21:20:34.317000",
          "content": "<p>Thank you!</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 393229,
      "author_name": "JohnM",
      "author_url": "",
      "post_date": "2018-09-25T02:29:06",
      "content": "<p>Thinking outside the box is one of the best things data scientists do when solving a problem. Many times a breakthrough comes from using additional data that no one previously considered. Transfer learning is pretty much built on this principle, as is knowledge sharing in general.</p>\n\n<p>I would find it helpful if someone could explain the rationale in preventing the use of \"external\" data.</p>",
      "votes": 4,
      "replies": [
        {
          "id": 393478,
          "author_name": "Allen",
          "author_url": "",
          "post_date": "2018-09-25T12:58:11.817000",
          "content": "<p>Imagine I work for a company that has proprietary data that is comparable to what is used in this competition. If I integrated that data into the training of my model, I would have a competitive advantage against others in this competition who don't have access to that same data. Many would argue that this is unfair, and I suppose that that is why the rule is in place.</p>\n\n<p>That said, I don't know why we shouldn't be allowed to use external data that is open source and available to everyone (e.g. census data). My only guess is that maybe it makes it harder for them to detect cheating.</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 393504,
          "author_name": "JohnM",
          "author_url": "",
          "post_date": "2018-09-25T13:38:37.307000",
          "content": "<p>Makes sense with proprietary data giving an advantage.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 393526,
          "author_name": "Leonid Gilburd",
          "author_url": "",
          "post_date": "2018-09-25T14:20:54.530000",
          "content": "<p>So maybe <a href=\"/philculliton\">@philculliton</a> can help us here? Can we use publically available external data? </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 393543,
          "author_name": "Fernando Constantino",
          "author_url": "",
          "post_date": "2018-09-25T14:42:09.850000",
          "content": "<p>Seeing the replies to Phil's last post, he may well be now taking a Sabbatical year on the North Pole..  Hope things quiet down with or without competition reset and we can move on.\nI would definitely allow external data as long as it is open source and is published on this forum.</p>",
          "votes": 3,
          "replies": []
        }
      ]
    },
    {
      "id": 420501,
      "author_name": "Ramson",
      "author_url": "",
      "post_date": "2018-11-13T18:20:57.763000",
      "content": "<p>External data of other of any structured data can help your to provide the part where it having the part which user actually needs you can access the other devices via <a href=\"http://www.121techsupport.co/garmim-map-update/\">http://www.121techsupport.co/garmim-map-update/</a>  and this is going to provide your the entire parts of structured data. </p>",
      "votes": -1,
      "replies": []
    },
    {
      "id": 392056,
      "author_name": "Brat Pat",
      "author_url": "",
      "post_date": "2018-09-22T23:06:09.453000",
      "content": "<p>The external data where it will be going to have the particular fact which is going to provide the part where it is havi9ng the part that the user needs to raise the external device through the <a href=\"https://hpsupports.co/hp-support-australia/\">https://hpsupports.co/hp-support-australia/</a> which is going to provide the part all around it </p>",
      "votes": -4,
      "replies": [
        {
          "id": 393221,
          "author_name": "JohnM",
          "author_url": "",
          "post_date": "2018-09-25T02:13:17.873000",
          "content": "<p>Hi Brat Pat. Why do you think Kagglers like to use external data? Also, how do you feel about Judg Kavanagh's prospects? </p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 393992,
      "author_name": "",
      "author_url": "",
      "post_date": "2018-09-26T07:02:31.067000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "391222": "The rules from the competition state the following:\n\n&gt; C. External Data. Unless otherwise expressly stated on the Competition Website, you may not use data other than the Competition Data to develop and test your models and Submissions. The Competition Sponsor reserves the right to disqualify any participant who Competition Sponsor discovers has used or attempted to use data other than the Competition Data in connection with the Competition, or who uses the Competition Data other than as permitted by the Competition Website and these Rules.\n\nHow strictly is this interpreted? For instance, we are given geographic data. If I replace the country values with something like the Index of Economic Freedom (which is an external but open data source), would that be considered in violation of the rules?",
    "391240": "Wondering the same but about time zone... Has anyone converted time to visitor's local time?",
    "393705": "We have agreed with the host to allow the use of **publicly, freely available data**, as long as the source of any external data is posted to the official competition forum prior to the First Submission Deadline. I will open a new thread shortly for official sharing of those external data sources.",
    "393229": "Thinking outside the box is one of the best things data scientists do when solving a problem. Many times a breakthrough comes from using additional data that no one previously considered. Transfer learning is pretty much built on this principle, as is knowledge sharing in general.\n\nI would find it helpful if someone could explain the rationale in preventing the use of \"external\" data.",
    "420501": "External data of other of any structured data can help your to provide the part where it having the part which user actually needs you can access the other devices via http://www.121techsupport.co/garmim-map-update/  and this is going to provide your the entire parts of structured data. ",
    "392056": "The external data where it will be going to have the particular fact which is going to provide the part where it is havi9ng the part that the user needs to raise the external device through the https://hpsupports.co/hp-support-australia/ which is going to provide the part all around it ",
    "393992": ""
  }
}