Can any one point me to kernel with data cleaning if there is any shown up ?
", "messages": [ { "id": "931242", "postDate": "07/16/2020 04:46:47", "content": "Can any one point me to kernel with data cleaning if there is any shown up ?
", "rawMarkdown": "Can any one point me to kernel with data cleaning if there is any shown up ?", "votes": null }, { "id": "931256", "postDate": "07/16/2020 05:01:29", "content": "From a quick look:\n- Augmentations, Data Cleaning and Bounding Boxes\n- cleaning and preprocessing train.csv\n- Check&Clean Big&Small bboxes
", "rawMarkdown": "From a quick look:\n- [Augmentations, Data Cleaning and Bounding Boxes](https://www.kaggle.com/reighns/augmentations-data-cleaning-and-bounding-boxes)\n- [cleaning and preprocessing train.csv](https://www.kaggle.com/ravi02516/cleaning-and-preprocessing-train-csv)\n- [Check&Clean Big&Small bboxes](https://www.kaggle.com/raininbox/check-clean-big-small-bboxes)", "votes": null }, { "id": "941444", "postDate": "07/23/2020 08:11:07", "content": "@matthewmasters thanks\n1)\nhttps://www.kaggle.com/ravi02516/cleaning-and-preprocessing-train-csv\nthis removes lot of bboxes\n\ntotal bboxes before cleaning 147793\ntotal bboxes after cleaning 138831\ntotal bboxes cleaned 8962\n
https://www.kaggle.com/raininbox/check-clean-big-small-bboxes\nThis removes only 216..
\n\nis 8962 a right number ? very confused which one to be used up
\n\nwhich version seem to be more correct one is here\nhttps://www.kaggle.com/c/global-wheat-detection/discussion/159578\nbad_boxes = [3687, 117344,173,113947,52868,2159,2169,121633,121634,147504,118211,52727,147552]
2) to comply with albumentation what is format of bbox to be used ?
", "rawMarkdown": "matthewmasters thanks\n1)\nhttps://www.kaggle.com/ravi02516/cleaning-and-preprocessing-train-csv\nthis removes lot of bboxes\n```\ntotal bboxes before cleaning 147793\ntotal bboxes after cleaning 138831\ntotal bboxes cleaned 8962\n```\n\nhttps://www.kaggle.com/raininbox/check-clean-big-small-bboxes\nThis removes only 216..\n\nis 8962 a right number ? very confused which one to be used up\n\nwhich version seem to be more correct one is here\nhttps://www.kaggle.com/c/global-wheat-detection/discussion/159578\n`bad_boxes = [3687, 117344,173,113947,52868,2159,2169,121633,121634,147504,118211,52727,147552]`\n\n\n\n2) to comply with albumentation what is format of bbox to be used ?", "votes": null }, { "id": "942884", "postDate": "07/24/2020 03:48:32", "content": "1) I have not used these notebooks so I don't know which one is more accurate. You will have to investigate that yourself (i.e. look at which boxes are removed and judge for yourself if they should be removed or not).
\n\n2) I believe albumentation supports multiple box formats. You provide the format like this:\n\nA.Compose(\n [\n ToTensorV2(p=1.0),\n ],\n bbox_params=A.BboxParams(\n format='pascal_voc',\n min_area=0,\n min_visibility=0,\n label_fields=['labels']\n )\n )\n
@matthewmasters many thanks \n .i figured it out by viewing \n few more things
\n\n1)Why in train kernels no one is doing Normalize Transform ,only min max scaling
\n\n2) During training i see that alex is training with 512 resized images during inference how are we adapting box x,y predicted on resized 512 to original image size of 1024
", "rawMarkdown": "matthewmasters many thanks \n .i figured it out by viewing \n few more things\n\n1)Why in train kernels no one is doing Normalize Transform ,only min max scaling\n\n2) During training i see that alex is training with 512 resized images during inference how are we adapting box x,y predicted on resized 512 to original image size of 1024", "votes": null }, { "id": "942951", "postDate": "07/24/2020 04:34:46", "content": "1) Some frameworks will normalize the data when input to the network. torchvision FasterRCNN for instance uses the GeneralizedRCNNTransform. Not sure about effdet
\n\n2) During training, albumentations will resize the box. During inference, he probably rescales the image, gets the prediction, then multiplies the result by 2
", "rawMarkdown": "1) Some frameworks will normalize the data when input to the network. torchvision FasterRCNN for instance uses the [GeneralizedRCNNTransform](https://github.com/pytorch/vision/blob/1aef87d01eec2c0989458387fa04baebcc86ea7b/torchvision/models/detection/transform.py#L58). Not sure about effdet\n\n2) During training, albumentations will resize the box. During inference, he probably rescales the image, gets the prediction, then multiplies the result by 2", "votes": null } ], "comments": [ { "id": 931256, "author_name": "matthewmasters", "author_url": "", "post_date": "07/16/2020 05:01:29", "content": "From a quick look:\n- Augmentations, Data Cleaning and Bounding Boxes\n- cleaning and preprocessing train.csv\n- Check&Clean Big&Small bboxes
", "votes": null, "replies": [ { "id": 941444, "author_name": "jaideepvalani", "author_url": "", "post_date": "07/23/2020 08:11:07", "content": "@matthewmasters thanks\n1)\nhttps://www.kaggle.com/ravi02516/cleaning-and-preprocessing-train-csv\nthis removes lot of bboxes\n\ntotal bboxes before cleaning 147793\ntotal bboxes after cleaning 138831\ntotal bboxes cleaned 8962\n
https://www.kaggle.com/raininbox/check-clean-big-small-bboxes\nThis removes only 216..
\n\nis 8962 a right number ? very confused which one to be used up
\n\nwhich version seem to be more correct one is here\nhttps://www.kaggle.com/c/global-wheat-detection/discussion/159578\nbad_boxes = [3687, 117344,173,113947,52868,2159,2169,121633,121634,147504,118211,52727,147552]
2) to comply with albumentation what is format of bbox to be used ?
", "votes": null, "replies": [] }, { "id": 942884, "author_name": "matthewmasters", "author_url": "", "post_date": "07/24/2020 03:48:32", "content": "1) I have not used these notebooks so I don't know which one is more accurate. You will have to investigate that yourself (i.e. look at which boxes are removed and judge for yourself if they should be removed or not).
\n\n2) I believe albumentation supports multiple box formats. You provide the format like this:\n\nA.Compose(\n [\n ToTensorV2(p=1.0),\n ],\n bbox_params=A.BboxParams(\n format='pascal_voc',\n min_area=0,\n min_visibility=0,\n label_fields=['labels']\n )\n )\n
@matthewmasters many thanks \n .i figured it out by viewing \n few more things
\n\n1)Why in train kernels no one is doing Normalize Transform ,only min max scaling
\n\n2) During training i see that alex is training with 512 resized images during inference how are we adapting box x,y predicted on resized 512 to original image size of 1024
", "votes": null, "replies": [] }, { "id": 942951, "author_name": "matthewmasters", "author_url": "", "post_date": "07/24/2020 04:34:46", "content": "1) Some frameworks will normalize the data when input to the network. torchvision FasterRCNN for instance uses the GeneralizedRCNNTransform. Not sure about effdet
\n\n2) During training, albumentations will resize the box. During inference, he probably rescales the image, gets the prediction, then multiplies the result by 2
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "931242": "Can any one point me to kernel with data cleaning if there is any shown up ?", "931256": "From a quick look:\n- [Augmentations, Data Cleaning and Bounding Boxes](https://www.kaggle.com/reighns/augmentations-data-cleaning-and-bounding-boxes)\n- [cleaning and preprocessing train.csv](https://www.kaggle.com/ravi02516/cleaning-and-preprocessing-train-csv)\n- [Check&Clean Big&Small bboxes](https://www.kaggle.com/raininbox/check-clean-big-small-bboxes)", "941444": "matthewmasters thanks\n1)\nhttps://www.kaggle.com/ravi02516/cleaning-and-preprocessing-train-csv\nthis removes lot of bboxes\n```\ntotal bboxes before cleaning 147793\ntotal bboxes after cleaning 138831\ntotal bboxes cleaned 8962\n```\n\nhttps://www.kaggle.com/raininbox/check-clean-big-small-bboxes\nThis removes only 216..\n\nis 8962 a right number ? very confused which one to be used up\n\nwhich version seem to be more correct one is here\nhttps://www.kaggle.com/c/global-wheat-detection/discussion/159578\n`bad_boxes = [3687, 117344,173,113947,52868,2159,2169,121633,121634,147504,118211,52727,147552]`\n\n\n\n2) to comply with albumentation what is format of bbox to be used ?", "942884": "1) I have not used these notebooks so I don't know which one is more accurate. You will have to investigate that yourself (i.e. look at which boxes are removed and judge for yourself if they should be removed or not).\n\n2) I believe albumentation supports multiple box formats. You provide the format like this:\n```\nA.Compose(\n [\n ToTensorV2(p=1.0),\n ],\n bbox_params=A.BboxParams(\n format='pascal_voc',\n min_area=0,\n min_visibility=0,\n label_fields=['labels']\n )\n )\n```", "942930": "matthewmasters many thanks \n .i figured it out by viewing \n few more things\n\n1)Why in train kernels no one is doing Normalize Transform ,only min max scaling\n\n2) During training i see that alex is training with 512 resized images during inference how are we adapting box x,y predicted on resized 512 to original image size of 1024", "942951": "1) Some frameworks will normalize the data when input to the network. torchvision FasterRCNN for instance uses the [GeneralizedRCNNTransform](https://github.com/pytorch/vision/blob/1aef87d01eec2c0989458387fa04baebcc86ea7b/torchvision/models/detection/transform.py#L58). Not sure about effdet\n\n2) During training, albumentations will resize the box. During inference, he probably rescales the image, gets the prediction, then multiplies the result by 2" }, "source": "meta" }