{
  "id": 323487,
  "title": "Are we allowed to use external data?",
  "url": "/competitions/denoising-shabby-pages/discussion/323487",
  "author_name": "",
  "post_date": "2022-05-06T20:23:42.581711700Z",
  "votes": null,
  "comment_count": 1,
  "views": 0,
  "content": "<p>How would we create our own data with normal documents?</p>",
  "messages": [
    {
      "id": "1779774",
      "postDate": "05/06/2022 20:23:42",
      "content": "<p>How would we create our own data with normal documents?</p>",
      "rawMarkdown": "How would we create our own data with normal documents?",
      "votes": null
    },
    {
      "id": "1779784",
      "postDate": "05/06/2022 20:28:23",
      "content": "<p>Yes, you're allowed and even encouraged to use external data. Feel free to use Augraphy to generate your own dataset; please share your experience with that in the discussion board and/or in slack if you do. The Augraphy pipeline used to create the ShabbyPages dataset is located at <a href=\"https://github.com/sparkfish/shabby-pages/blob/dev/shabbypipeline.py\" target=\"_blank\">https://github.com/sparkfish/shabby-pages/blob/dev/shabbypipeline.py</a>. You will probably have to play around with the pipeline a bit to fit your system. If you need documents to run through the pipeline, we have a corpus of clean documents to use at <a href=\"https://drive.google.com/drive/folders/1NJhrxo5wyQnyPUKuRkIMOmcW7Zkd2J_0?usp=sharing\" target=\"_blank\">https://drive.google.com/drive/folders/1NJhrxo5wyQnyPUKuRkIMOmcW7Zkd2J_0?usp=sharing</a>. Let us know if you have any other questions!</p>",
      "rawMarkdown": "Yes, you're allowed and even encouraged to use external data. Feel free to use Augraphy to generate your own dataset; please share your experience with that in the discussion board and/or in slack if you do. The Augraphy pipeline used to create the ShabbyPages dataset is located at [https://github.com/sparkfish/shabby-pages/blob/dev/shabbypipeline.py](https://github.com/sparkfish/shabby-pages/blob/dev/shabbypipeline.py). You will probably have to play around with the pipeline a bit to fit your system. If you need documents to run through the pipeline, we have a corpus of clean documents to use at [https://drive.google.com/drive/folders/1NJhrxo5wyQnyPUKuRkIMOmcW7Zkd2J_0?usp=sharing](https://drive.google.com/drive/folders/1NJhrxo5wyQnyPUKuRkIMOmcW7Zkd2J_0?usp=sharing). Let us know if you have any other questions!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1779784,
      "author_name": "augraphy",
      "author_url": "",
      "post_date": "05/06/2022 20:28:23",
      "content": "<p>Yes, you're allowed and even encouraged to use external data. Feel free to use Augraphy to generate your own dataset; please share your experience with that in the discussion board and/or in slack if you do. The Augraphy pipeline used to create the ShabbyPages dataset is located at <a href=\"https://github.com/sparkfish/shabby-pages/blob/dev/shabbypipeline.py\" target=\"_blank\">https://github.com/sparkfish/shabby-pages/blob/dev/shabbypipeline.py</a>. You will probably have to play around with the pipeline a bit to fit your system. If you need documents to run through the pipeline, we have a corpus of clean documents to use at <a href=\"https://drive.google.com/drive/folders/1NJhrxo5wyQnyPUKuRkIMOmcW7Zkd2J_0?usp=sharing\" target=\"_blank\">https://drive.google.com/drive/folders/1NJhrxo5wyQnyPUKuRkIMOmcW7Zkd2J_0?usp=sharing</a>. Let us know if you have any other questions!</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1779774": "How would we create our own data with normal documents?",
    "1779784": "Yes, you're allowed and even encouraged to use external data. Feel free to use Augraphy to generate your own dataset; please share your experience with that in the discussion board and/or in slack if you do. The Augraphy pipeline used to create the ShabbyPages dataset is located at [https://github.com/sparkfish/shabby-pages/blob/dev/shabbypipeline.py](https://github.com/sparkfish/shabby-pages/blob/dev/shabbypipeline.py). You will probably have to play around with the pipeline a bit to fit your system. If you need documents to run through the pipeline, we have a corpus of clean documents to use at [https://drive.google.com/drive/folders/1NJhrxo5wyQnyPUKuRkIMOmcW7Zkd2J_0?usp=sharing](https://drive.google.com/drive/folders/1NJhrxo5wyQnyPUKuRkIMOmcW7Zkd2J_0?usp=sharing). Let us know if you have any other questions!"
  },
  "source": "meta"
}