{
  "id": 237364,
  "title": "Winning Solution Design Discussion",
  "url": "/competitions/coleridgeinitiative-show-us-the-data/discussion/237364",
  "author_name": "",
  "post_date": "2021-05-08T11:54:09.583297900Z",
  "votes": 12,
  "comment_count": 3,
  "views": 0,
  "content": "<p>I think the reason that prediction models are not working very well, among other reasons, is because the challenge calls for:</p>\n<p>\"Your predictions will be short excerpts from the publications that appear to note a dataset.\"</p>\n<p>So when you use matching to determine if a string \"is in\" a text, if it is, then you return the actual words from the text as your prediction - because the dataset title words are actually in the text.  If you are returning a label from the dataset, that is not going to receive as good of score, because it is not actually what is in the text and is not the challenge and your lables to predict are limited.  Plus you have no labels for unseen datasets.</p>\n<p>So IMO, a winning models has to, at a minimum:</p>\n<ol>\n<li><p>be able to locate all likely govt' dataset titles in every document (previously seen or unseen)</p></li>\n<li><p>extract the actual text that is presumed to be the location of the dataset title and return it as the prediction.</p></li>\n</ol>\n<p>Simple, right?</p>",
  "messages": [
    {
      "id": "1297861",
      "postDate": "05/08/2021 11:54:09",
      "content": "<p>I think the reason that prediction models are not working very well, among other reasons, is because the challenge calls for:</p>\n<p>\"Your predictions will be short excerpts from the publications that appear to note a dataset.\"</p>\n<p>So when you use matching to determine if a string \"is in\" a text, if it is, then you return the actual words from the text as your prediction - because the dataset title words are actually in the text.  If you are returning a label from the dataset, that is not going to receive as good of score, because it is not actually what is in the text and is not the challenge and your lables to predict are limited.  Plus you have no labels for unseen datasets.</p>\n<p>So IMO, a winning models has to, at a minimum:</p>\n<ol>\n<li><p>be able to locate all likely govt' dataset titles in every document (previously seen or unseen)</p></li>\n<li><p>extract the actual text that is presumed to be the location of the dataset title and return it as the prediction.</p></li>\n</ol>\n<p>Simple, right?</p>",
      "rawMarkdown": "I think the reason that prediction models are not working very well, among other reasons, is because the challenge calls for:\n\n\"Your predictions will be short excerpts from the publications that appear to note a dataset.\"\n\nSo when you use matching to determine if a string \"is in\" a text, if it is, then you return the actual words from the text as your prediction - because the dataset title words are actually in the text.  If you are returning a label from the dataset, that is not going to receive as good of score, because it is not actually what is in the text and is not the challenge and your lables to predict are limited.  Plus you have no labels for unseen datasets.\n\nSo IMO, a winning models has to, at a minimum:\n\n1. be able to locate all likely govt' dataset titles in every document (previously seen or unseen)\n\n2. extract the actual text that is presumed to be the location of the dataset title and return it as the prediction.\n\nSimple, right?",
      "votes": null
    },
    {
      "id": "1300938",
      "postDate": "05/10/2021 21:05:13",
      "content": "<p>I've done that. It's just that according to my calculations, if the internal test set is ~8000, my solution will need 20-30 hours to complete. So now I try to trim it down, which gives me many headaches -_-</p>",
      "rawMarkdown": "I've done that. It's just that according to my calculations, if the internal test set is ~8000, my solution will need 20-30 hours to complete. So now I try to trim it down, which gives me many headaches -_-",
      "votes": null
    },
    {
      "id": "1302197",
      "postDate": "05/11/2021 12:39:14",
      "content": "<p>The submission is actually faster than exptected: I simulated a submission run with 7000-8000 publication in the training data and it took about 7 hours.. when submitting the same model it only took about three hours to run through..</p>",
      "rawMarkdown": "The submission is actually faster than exptected: I simulated a submission run with 7000-8000 publication in the training data and it took about 7 hours.. when submitting the same model it only took about three hours to run through..",
      "votes": null
    },
    {
      "id": "1302537",
      "postDate": "05/11/2021 15:31:55",
      "content": "<p>Thanks for the insight! I might be closer than I thought then</p>",
      "rawMarkdown": "Thanks for the insight! I might be closer than I thought then",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1300938,
      "author_name": "aglaros",
      "author_url": "",
      "post_date": "05/10/2021 21:05:13",
      "content": "<p>I've done that. It's just that according to my calculations, if the internal test set is ~8000, my solution will need 20-30 hours to complete. So now I try to trim it down, which gives me many headaches -_-</p>",
      "votes": null,
      "replies": [
        {
          "id": 1302197,
          "author_name": "hannes82",
          "author_url": "",
          "post_date": "05/11/2021 12:39:14",
          "content": "<p>The submission is actually faster than exptected: I simulated a submission run with 7000-8000 publication in the training data and it took about 7 hours.. when submitting the same model it only took about three hours to run through..</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1302537,
          "author_name": "aglaros",
          "author_url": "",
          "post_date": "05/11/2021 15:31:55",
          "content": "<p>Thanks for the insight! I might be closer than I thought then</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1297861": "I think the reason that prediction models are not working very well, among other reasons, is because the challenge calls for:\n\n\"Your predictions will be short excerpts from the publications that appear to note a dataset.\"\n\nSo when you use matching to determine if a string \"is in\" a text, if it is, then you return the actual words from the text as your prediction - because the dataset title words are actually in the text.  If you are returning a label from the dataset, that is not going to receive as good of score, because it is not actually what is in the text and is not the challenge and your lables to predict are limited.  Plus you have no labels for unseen datasets.\n\nSo IMO, a winning models has to, at a minimum:\n\n1. be able to locate all likely govt' dataset titles in every document (previously seen or unseen)\n\n2. extract the actual text that is presumed to be the location of the dataset title and return it as the prediction.\n\nSimple, right?",
    "1300938": "I've done that. It's just that according to my calculations, if the internal test set is ~8000, my solution will need 20-30 hours to complete. So now I try to trim it down, which gives me many headaches -_-",
    "1302197": "The submission is actually faster than exptected: I simulated a submission run with 7000-8000 publication in the training data and it took about 7 hours.. when submitting the same model it only took about three hours to run through..",
    "1302537": "Thanks for the insight! I might be closer than I thought then"
  },
  "source": "meta"
}