{
  "id": 162899,
  "title": "Data Preprocessing for the TensorflowAPI",
  "url": "/competitions/global-wheat-detection/discussion/162899",
  "author_name": "",
  "post_date": "2020-06-30T11:27:39.064338200Z",
  "votes": 1,
  "comment_count": 4,
  "views": 0,
  "content": "<p>Here, I prepared a Notebook which helps us to use in the TensorflowAPI</p>\n\n<p>Data was converted into the CSV file </p>\n\n<p>What's next?</p>\n\n<ol>\n<li>Take CSV file sperate into CSV Train, and CSV Test</li>\n<li>Convert CSV files into the tf.records file </li>\n<li>Then use for the training</li>\n</ol>\n\n<p>Happy learning :)</p>\n\n<p>Hope it is useful for you \nIF YOU LIKE IT UPVOTE FOR ME</p>\n\n<p><code>import numpy as np\nimport pandas as pd</code></p>\n\n<p>Importing Numpy and Pandas files</p>\n\n<p><code>data = pd.read_csv(\"train.csv\")\ndata.head()</code></p>\n\n<p><code>bbox = data['bbox']</code> </p>\n\n<p><code>col0 = []\ncol1 = []\ncol2 = []\ncol3 = []</code></p>\n\n<p><code>for i in range(len(bbox)):\n    res_bb = bbox[i].strip('][').split(', ')\n    for j in range(len(res_bb)):\n        res_bb[j] = float(res_bb[j])</code></p>\n\n<pre><code>   `if j == 0:\n        col0.append(res_bb[j])\n    elif j == 1:\n        col1.append(res_bb[j])\n    elif j == 2:\n        col2.append(res_bb[j])\n    elif j == 3:\n        col3.append(res_bb[j])\n    else:\n        print(\"Hoooo\")`\n</code></pre>\n\n<p>converting string list to the Normal list and then creating the columns for xmin, ymin, xmax, ymax</p>\n\n<p><code>from sklearn.preprocessing import LabelEncoder\nlb = LabelEncoder()\ndata['source'] = lb.fit_transform(data['source'])\nsource = data['source']\nset(source)</code></p>\n\n<p>Encoding the data from characters to numbers</p>\n\n<p>`data.drop([\"bbox\", 'source'], axis=1, inplace=True)</p>\n\n<p>data['class'] = source\ndata['xmin'] = col0\ndata['ymin'] = col1\ndata['xmax'] = col2\ndata['ymax'] = col3`</p>\n\n<p>dropping unwanted data and assining the new columns</p>\n\n<p><code>data = data.rename(columns={'image_id':'filename'})</code></p>\n\n<p>renaming the first column to make convininent to the tensorflowapi</p>\n\n<p>`import os</p>\n\n<p>data.to_csv(os.getcwd()+\"/finaldata.csv\", index=None)`</p>\n\n<p>saving the new data</p>\n\n<p><code>d = pd.read_csv('finaldata.csv')\nd.head()</code></p>\n\n<p>Checking the data</p>\n\n<p>You can find the ipynb file below</p>\n\n<p>Hope it is help ful to you, upvote for this,</p>\n\n<p>Thank you</p>",
  "messages": [
    {
      "id": "908106",
      "postDate": "06/30/2020 11:27:39",
      "content": "<p>Here, I prepared a Notebook which helps us to use in the TensorflowAPI</p>\n\n<p>Data was converted into the CSV file </p>\n\n<p>What's next?</p>\n\n<ol>\n<li>Take CSV file sperate into CSV Train, and CSV Test</li>\n<li>Convert CSV files into the tf.records file </li>\n<li>Then use for the training</li>\n</ol>\n\n<p>Happy learning :)</p>\n\n<p>Hope it is useful for you \nIF YOU LIKE IT UPVOTE FOR ME</p>\n\n<p><code>import numpy as np\nimport pandas as pd</code></p>\n\n<p>Importing Numpy and Pandas files</p>\n\n<p><code>data = pd.read_csv(\"train.csv\")\ndata.head()</code></p>\n\n<p><code>bbox = data['bbox']</code> </p>\n\n<p><code>col0 = []\ncol1 = []\ncol2 = []\ncol3 = []</code></p>\n\n<p><code>for i in range(len(bbox)):\n    res_bb = bbox[i].strip('][').split(', ')\n    for j in range(len(res_bb)):\n        res_bb[j] = float(res_bb[j])</code></p>\n\n<pre><code>   `if j == 0:\n        col0.append(res_bb[j])\n    elif j == 1:\n        col1.append(res_bb[j])\n    elif j == 2:\n        col2.append(res_bb[j])\n    elif j == 3:\n        col3.append(res_bb[j])\n    else:\n        print(\"Hoooo\")`\n</code></pre>\n\n<p>converting string list to the Normal list and then creating the columns for xmin, ymin, xmax, ymax</p>\n\n<p><code>from sklearn.preprocessing import LabelEncoder\nlb = LabelEncoder()\ndata['source'] = lb.fit_transform(data['source'])\nsource = data['source']\nset(source)</code></p>\n\n<p>Encoding the data from characters to numbers</p>\n\n<p>`data.drop([\"bbox\", 'source'], axis=1, inplace=True)</p>\n\n<p>data['class'] = source\ndata['xmin'] = col0\ndata['ymin'] = col1\ndata['xmax'] = col2\ndata['ymax'] = col3`</p>\n\n<p>dropping unwanted data and assining the new columns</p>\n\n<p><code>data = data.rename(columns={'image_id':'filename'})</code></p>\n\n<p>renaming the first column to make convininent to the tensorflowapi</p>\n\n<p>`import os</p>\n\n<p>data.to_csv(os.getcwd()+\"/finaldata.csv\", index=None)`</p>\n\n<p>saving the new data</p>\n\n<p><code>d = pd.read_csv('finaldata.csv')\nd.head()</code></p>\n\n<p>Checking the data</p>\n\n<p>You can find the ipynb file below</p>\n\n<p>Hope it is help ful to you, upvote for this,</p>\n\n<p>Thank you</p>",
      "rawMarkdown": "Here, I prepared a Notebook which helps us to use in the TensorflowAPI\n\nData was converted into the CSV file \n\nWhat's next?\n\n1. Take CSV file sperate into CSV Train, and CSV Test\n2. Convert CSV files into the tf.records file \n3. Then use for the training\n\n\nHappy learning :)\n\nHope it is useful for you \nIF YOU LIKE IT UPVOTE FOR ME\n\n\n\n\n\n\n\n\n\n`import numpy as np\nimport pandas as pd`\n\nImporting Numpy and Pandas files\n\n`data = pd.read_csv(\"train.csv\")\ndata.head()`\n\n`bbox = data['bbox']` \n\n`col0 = []\ncol1 = []\ncol2 = []\ncol3 = []`\n\n`for i in range(len(bbox)):\n    res_bb = bbox[i].strip('][').split(', ')\n    for j in range(len(res_bb)):\n        res_bb[j] = float(res_bb[j])`\n      \n       `if j == 0:\n            col0.append(res_bb[j])\n        elif j == 1:\n            col1.append(res_bb[j])\n        elif j == 2:\n            col2.append(res_bb[j])\n        elif j == 3:\n            col3.append(res_bb[j])\n        else:\n            print(\"Hoooo\")`\n\nconverting string list to the Normal list and then creating the columns for xmin, ymin, xmax, ymax\n\n\n`from sklearn.preprocessing import LabelEncoder\nlb = LabelEncoder()\ndata['source'] = lb.fit_transform(data['source'])\nsource = data['source']\nset(source)`\n\nEncoding the data from characters to numbers\n\n\n`data.drop([\"bbox\", 'source'], axis=1, inplace=True)\n\ndata['class'] = source\ndata['xmin'] = col0\ndata['ymin'] = col1\ndata['xmax'] = col2\ndata['ymax'] = col3`\n\ndropping unwanted data and assining the new columns\n\n`data = data.rename(columns={'image_id':'filename'})`\n\nrenaming the first column to make convininent to the tensorflowapi\n\n`import os\n\ndata.to_csv(os.getcwd()+\"/finaldata.csv\", index=None)`\n\nsaving the new data\n\n`d = pd.read_csv('finaldata.csv')\nd.head()`\n\nChecking the data\n\n\nYou can find the ipynb file below\n\nHope it is help ful to you, upvote for this,\n\nThank you",
      "votes": null
    },
    {
      "id": "908243",
      "postDate": "06/30/2020 13:13:10",
      "content": "<p>Looks pretty neat 👌</p>",
      "rawMarkdown": "Looks pretty neat 👌",
      "votes": null
    },
    {
      "id": "909502",
      "postDate": "06/30/2020 16:13:22",
      "content": "<p>Thank you, very much </p>\n\n<p>please upvote if you like</p>",
      "rawMarkdown": "Thank you, very much \n\nplease upvote if you like",
      "votes": null
    },
    {
      "id": "909564",
      "postDate": "06/30/2020 16:57:00",
      "content": "<p>already done :')</p>",
      "rawMarkdown": "already done :')",
      "votes": null
    },
    {
      "id": "909566",
      "postDate": "06/30/2020 16:58:07",
      "content": "<p><a href=\"/saichandrareddy\">@saichandrareddy</a>  your linking is wrong though</p>",
      "rawMarkdown": "saichandrareddy  your linking is wrong though",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 908243,
      "author_name": "yash612",
      "author_url": "",
      "post_date": "06/30/2020 13:13:10",
      "content": "<p>Looks pretty neat 👌</p>",
      "votes": null,
      "replies": [
        {
          "id": 909502,
          "author_name": "saichandrareddy",
          "author_url": "",
          "post_date": "06/30/2020 16:13:22",
          "content": "<p>Thank you, very much </p>\n\n<p>please upvote if you like</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 909564,
          "author_name": "yash612",
          "author_url": "",
          "post_date": "06/30/2020 16:57:00",
          "content": "<p>already done :')</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 909566,
          "author_name": "yash612",
          "author_url": "",
          "post_date": "06/30/2020 16:58:07",
          "content": "<p><a href=\"/saichandrareddy\">@saichandrareddy</a>  your linking is wrong though</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "908106": "Here, I prepared a Notebook which helps us to use in the TensorflowAPI\n\nData was converted into the CSV file \n\nWhat's next?\n\n1. Take CSV file sperate into CSV Train, and CSV Test\n2. Convert CSV files into the tf.records file \n3. Then use for the training\n\n\nHappy learning :)\n\nHope it is useful for you \nIF YOU LIKE IT UPVOTE FOR ME\n\n\n\n\n\n\n\n\n\n`import numpy as np\nimport pandas as pd`\n\nImporting Numpy and Pandas files\n\n`data = pd.read_csv(\"train.csv\")\ndata.head()`\n\n`bbox = data['bbox']` \n\n`col0 = []\ncol1 = []\ncol2 = []\ncol3 = []`\n\n`for i in range(len(bbox)):\n    res_bb = bbox[i].strip('][').split(', ')\n    for j in range(len(res_bb)):\n        res_bb[j] = float(res_bb[j])`\n      \n       `if j == 0:\n            col0.append(res_bb[j])\n        elif j == 1:\n            col1.append(res_bb[j])\n        elif j == 2:\n            col2.append(res_bb[j])\n        elif j == 3:\n            col3.append(res_bb[j])\n        else:\n            print(\"Hoooo\")`\n\nconverting string list to the Normal list and then creating the columns for xmin, ymin, xmax, ymax\n\n\n`from sklearn.preprocessing import LabelEncoder\nlb = LabelEncoder()\ndata['source'] = lb.fit_transform(data['source'])\nsource = data['source']\nset(source)`\n\nEncoding the data from characters to numbers\n\n\n`data.drop([\"bbox\", 'source'], axis=1, inplace=True)\n\ndata['class'] = source\ndata['xmin'] = col0\ndata['ymin'] = col1\ndata['xmax'] = col2\ndata['ymax'] = col3`\n\ndropping unwanted data and assining the new columns\n\n`data = data.rename(columns={'image_id':'filename'})`\n\nrenaming the first column to make convininent to the tensorflowapi\n\n`import os\n\ndata.to_csv(os.getcwd()+\"/finaldata.csv\", index=None)`\n\nsaving the new data\n\n`d = pd.read_csv('finaldata.csv')\nd.head()`\n\nChecking the data\n\n\nYou can find the ipynb file below\n\nHope it is help ful to you, upvote for this,\n\nThank you",
    "908243": "Looks pretty neat 👌",
    "909502": "Thank you, very much \n\nplease upvote if you like",
    "909564": "already done :')",
    "909566": "saichandrareddy  your linking is wrong though"
  },
  "source": "meta"
}