{
  "id": 178863,
  "title": "Extensive data",
  "url": "/competitions/gan-getting-started/discussion/178863",
  "author_name": "",
  "post_date": "2020-08-31T16:45:14.133589700Z",
  "votes": 4,
  "comment_count": 2,
  "views": 0,
  "content": "<p>300 images of monet is not enough. So I created a larger monet dataset by uploading an image from wikiart 1367images. And then convert it to TFRecords. Hope it helps your work</p>\n<p>Notebooks: <a href=\"https://www.kaggle.com/doanquanvietnamca/tfrecords-create?scriptVersionId=41726105\" target=\"_blank\">https://www.kaggle.com/doanquanvietnamca/tfrecords-create?scriptVersionId=41726105</a></p>\n<p>Dataset: <a href=\"https://www.kaggle.com/doanquanvietnamca/monet-tfrecords-extdata\" target=\"_blank\">https://www.kaggle.com/doanquanvietnamca/monet-tfrecords-extdata</a></p>",
  "messages": [
    {
      "id": "993095",
      "postDate": "08/31/2020 16:45:14",
      "content": "<p>300 images of monet is not enough. So I created a larger monet dataset by uploading an image from wikiart 1367images. And then convert it to TFRecords. Hope it helps your work</p>\n<p>Notebooks: <a href=\"https://www.kaggle.com/doanquanvietnamca/tfrecords-create?scriptVersionId=41726105\" target=\"_blank\">https://www.kaggle.com/doanquanvietnamca/tfrecords-create?scriptVersionId=41726105</a></p>\n<p>Dataset: <a href=\"https://www.kaggle.com/doanquanvietnamca/monet-tfrecords-extdata\" target=\"_blank\">https://www.kaggle.com/doanquanvietnamca/monet-tfrecords-extdata</a></p>",
      "rawMarkdown": "300 images of monet is not enough. So I created a larger monet dataset by uploading an image from wikiart 1367images. And then convert it to TFRecords. Hope it helps your work\n\nNotebooks: https://www.kaggle.com/doanquanvietnamca/tfrecords-create?scriptVersionId=41726105\n\nDataset: https://www.kaggle.com/doanquanvietnamca/monet-tfrecords-extdata",
      "votes": null
    },
    {
      "id": "993364",
      "postDate": "08/31/2020 21:36:27",
      "content": "<p>Thanks <a href=\"https://www.kaggle.com/doanquanvietnamca\" target=\"_blank\">@doanquanvietnamca</a> I was also looking for more data, did you scraped, or you downloaded from somewhere?</p>",
      "rawMarkdown": "Thanks @doanquanvietnamca I was also looking for more data, did you scraped, or you downloaded from somewhere?",
      "votes": null
    },
    {
      "id": "993367",
      "postDate": "08/31/2020 21:45:58",
      "content": "<p>Here is function to crab dataset:</p>\n<pre><code>def provide_monet_dataset():\n    # Dowload the script that will manage the data procurement\n    # https://github.com/lucasdavid/wikiart\n    os.system(\"git clone https://github.com/lucasdavid/wikiart.git\")\n    os.system(\"python3 wikiart/wikiart.py --datadir ./wikiart-saved/ fetch --only artists\")\n\n    # Metadata to indicate the artist to be downloaded\n    entry = '''[{\n        \"artistName\": \"Claude Monet\",\n        \"birthDay\": \"/Date(-4074969600000)/\",\n        \"birthDayAsString\": \"November 14, 1840\",\n        \"contentId\": 211667,\n        \"deathDay\": \"/Date(-1359331200000)/\",\n        \"deathDayAsString\": \"December 5, 1926\",\n        \"dictonaries\": [\n            1221,\n            316\n        ],\n        \"image\": \"https://uploads0.wikiart.org/00115/images/claude-monet/440px-claude-monet-1899-nadar-crop.jpg!Portrait.jpg\",\n        \"lastNameFirst\": \"Monet Claude\",\n        \"url\": \"claude-monet\",\n        \"wikipediaUrl\": \"http://en.wikipedia.org/wiki/Claude_Monet\"\n    }]'''\n\n    with open('./wikiart-saved/meta/artists.json', 'w') as artists_file:\n        artists_file.write(entry)\n\n    # Download the art from the artist\n    os.system(\"python3 wikiart/wikiart.py --datadir ./wikiart-saved/ fetch\")\n\n    # Filepath of all files retrieval\n    files = []\n    # r=root, d=directories, f = files\n    for r, d, f in os.walk('wikiart-saved/images/claude-monet'):\n        for file in f:\n            if '.jpg' in file:\n                files.append(os.path.join(r, file))\n\n    images = []\n\n    for file in files:\n        images.append(_preprocess(cv2.imread(file)))\n\n    # An array with al the read fieles is returned\n    return images\n</code></pre>",
      "rawMarkdown": "Here is function to crab dataset:\n\n```\ndef provide_monet_dataset():\n    # Dowload the script that will manage the data procurement\n    # https://github.com/lucasdavid/wikiart\n    os.system(\"git clone https://github.com/lucasdavid/wikiart.git\")\n    os.system(\"python3 wikiart/wikiart.py --datadir ./wikiart-saved/ fetch --only artists\")\n\n    # Metadata to indicate the artist to be downloaded\n    entry = '''[{\n        \"artistName\": \"Claude Monet\",\n        \"birthDay\": \"/Date(-4074969600000)/\",\n        \"birthDayAsString\": \"November 14, 1840\",\n        \"contentId\": 211667,\n        \"deathDay\": \"/Date(-1359331200000)/\",\n        \"deathDayAsString\": \"December 5, 1926\",\n        \"dictonaries\": [\n            1221,\n            316\n        ],\n        \"image\": \"https://uploads0.wikiart.org/00115/images/claude-monet/440px-claude-monet-1899-nadar-crop.jpg!Portrait.jpg\",\n        \"lastNameFirst\": \"Monet Claude\",\n        \"url\": \"claude-monet\",\n        \"wikipediaUrl\": \"http://en.wikipedia.org/wiki/Claude_Monet\"\n    }]'''\n\n    with open('./wikiart-saved/meta/artists.json', 'w') as artists_file:\n        artists_file.write(entry)\n\n    # Download the art from the artist\n    os.system(\"python3 wikiart/wikiart.py --datadir ./wikiart-saved/ fetch\")\n\n    # Filepath of all files retrieval\n    files = []\n    # r=root, d=directories, f = files\n    for r, d, f in os.walk('wikiart-saved/images/claude-monet'):\n        for file in f:\n            if '.jpg' in file:\n                files.append(os.path.join(r, file))\n\n    images = []\n\n    for file in files:\n        images.append(_preprocess(cv2.imread(file)))\n\n    # An array with al the read fieles is returned\n    return images\n```",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 993364,
      "author_name": "dimitreoliveira",
      "author_url": "",
      "post_date": "08/31/2020 21:36:27",
      "content": "<p>Thanks <a href=\"https://www.kaggle.com/doanquanvietnamca\" target=\"_blank\">@doanquanvietnamca</a> I was also looking for more data, did you scraped, or you downloaded from somewhere?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 993367,
      "author_name": "doanquanvietnamca",
      "author_url": "",
      "post_date": "08/31/2020 21:45:58",
      "content": "<p>Here is function to crab dataset:</p>\n<pre><code>def provide_monet_dataset():\n    # Dowload the script that will manage the data procurement\n    # https://github.com/lucasdavid/wikiart\n    os.system(\"git clone https://github.com/lucasdavid/wikiart.git\")\n    os.system(\"python3 wikiart/wikiart.py --datadir ./wikiart-saved/ fetch --only artists\")\n\n    # Metadata to indicate the artist to be downloaded\n    entry = '''[{\n        \"artistName\": \"Claude Monet\",\n        \"birthDay\": \"/Date(-4074969600000)/\",\n        \"birthDayAsString\": \"November 14, 1840\",\n        \"contentId\": 211667,\n        \"deathDay\": \"/Date(-1359331200000)/\",\n        \"deathDayAsString\": \"December 5, 1926\",\n        \"dictonaries\": [\n            1221,\n            316\n        ],\n        \"image\": \"https://uploads0.wikiart.org/00115/images/claude-monet/440px-claude-monet-1899-nadar-crop.jpg!Portrait.jpg\",\n        \"lastNameFirst\": \"Monet Claude\",\n        \"url\": \"claude-monet\",\n        \"wikipediaUrl\": \"http://en.wikipedia.org/wiki/Claude_Monet\"\n    }]'''\n\n    with open('./wikiart-saved/meta/artists.json', 'w') as artists_file:\n        artists_file.write(entry)\n\n    # Download the art from the artist\n    os.system(\"python3 wikiart/wikiart.py --datadir ./wikiart-saved/ fetch\")\n\n    # Filepath of all files retrieval\n    files = []\n    # r=root, d=directories, f = files\n    for r, d, f in os.walk('wikiart-saved/images/claude-monet'):\n        for file in f:\n            if '.jpg' in file:\n                files.append(os.path.join(r, file))\n\n    images = []\n\n    for file in files:\n        images.append(_preprocess(cv2.imread(file)))\n\n    # An array with al the read fieles is returned\n    return images\n</code></pre>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "993095": "300 images of monet is not enough. So I created a larger monet dataset by uploading an image from wikiart 1367images. And then convert it to TFRecords. Hope it helps your work\n\nNotebooks: https://www.kaggle.com/doanquanvietnamca/tfrecords-create?scriptVersionId=41726105\n\nDataset: https://www.kaggle.com/doanquanvietnamca/monet-tfrecords-extdata",
    "993364": "Thanks @doanquanvietnamca I was also looking for more data, did you scraped, or you downloaded from somewhere?",
    "993367": "Here is function to crab dataset:\n\n```\ndef provide_monet_dataset():\n    # Dowload the script that will manage the data procurement\n    # https://github.com/lucasdavid/wikiart\n    os.system(\"git clone https://github.com/lucasdavid/wikiart.git\")\n    os.system(\"python3 wikiart/wikiart.py --datadir ./wikiart-saved/ fetch --only artists\")\n\n    # Metadata to indicate the artist to be downloaded\n    entry = '''[{\n        \"artistName\": \"Claude Monet\",\n        \"birthDay\": \"/Date(-4074969600000)/\",\n        \"birthDayAsString\": \"November 14, 1840\",\n        \"contentId\": 211667,\n        \"deathDay\": \"/Date(-1359331200000)/\",\n        \"deathDayAsString\": \"December 5, 1926\",\n        \"dictonaries\": [\n            1221,\n            316\n        ],\n        \"image\": \"https://uploads0.wikiart.org/00115/images/claude-monet/440px-claude-monet-1899-nadar-crop.jpg!Portrait.jpg\",\n        \"lastNameFirst\": \"Monet Claude\",\n        \"url\": \"claude-monet\",\n        \"wikipediaUrl\": \"http://en.wikipedia.org/wiki/Claude_Monet\"\n    }]'''\n\n    with open('./wikiart-saved/meta/artists.json', 'w') as artists_file:\n        artists_file.write(entry)\n\n    # Download the art from the artist\n    os.system(\"python3 wikiart/wikiart.py --datadir ./wikiart-saved/ fetch\")\n\n    # Filepath of all files retrieval\n    files = []\n    # r=root, d=directories, f = files\n    for r, d, f in os.walk('wikiart-saved/images/claude-monet'):\n        for file in f:\n            if '.jpg' in file:\n                files.append(os.path.join(r, file))\n\n    images = []\n\n    for file in files:\n        images.append(_preprocess(cv2.imread(file)))\n\n    # An array with al the read fieles is returned\n    return images\n```"
  },
  "source": "meta"
}