![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F836513%2F370f7027f953a5885b223bd99e2301ae%2Ffold1.png?generation=1588684624055121&alt=media\")
As @stavness mentioned in the welcome post, the test images are from different regions than training images:\n> blind test set of images from different world regions than in the training set
\n\nWe find images from 7 different regions in training dataset.
\n\n> 'arvalis_1', 'arvalis_2', 'arvalis_3', 'ethz_1', 'inrae_1', 'rres_1', 'usask_1'
\n\nThus, kRegionsOut validation strategies in training is useful to tackle test images from other regions.
\n\ncode snippet:\n> df = pd.read_csv('../input/global-wheat-detection/train.csv')\n> df.head()\n
Now mapping source to fold\n> mapping = {'arvalis_1': 0, 'arvalis_2': 1, 'arvalis_3': 2, 'ethz_1': 3, 'inrae_1': 4, 'rres_1': 5, 'usask_1': 6}
\n\n> df['folds'] = df.replace({'source': mapping}).source.values\ndf.head()\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F836513%2Ff11da5880bb0a508df3524527163de91%2Ffold2.png?generation=1588684899683123&alt=media\")
Number of unique images by source\n> df.groupby('source')['image_id'].nunique()\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F836513%2Fcebe9252982e7e8483287cf97ab27a82%2Ffold8.png?generation=1588685722012651&alt=media\")
Number of unique images by fold\n> df.groupby('folds')['image_id'].nunique()\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F836513%2F4843ece74f13e080cd314344577ec04a%2Ffold7.png?generation=1588685706332999&alt=media\")
To get 'arvalis_1' region in validation set and all other regions in training:
\n\n\n", "messages": [ { "id": 834102, "postDate": "2020-05-05T09:53:12.387Z", "content": "train_ids = df.loc[df.folds.values!=0,'image_id'].unique().tolist()\n valid_ids = df.loc[df.folds.values==0,'image_id'].unique().tolist()
\n \ntrain_df = df[df['image_id'].isin(train_ids)]
\n \nvalid_df = df[df['image_id'].isin(valid_ids)]
\n
As @stavness mentioned in the welcome post, the test images are from different regions than training images:\n> blind test set of images from different world regions than in the training set
\n\nWe find images from 7 different regions in training dataset.
\n\n> 'arvalis_1', 'arvalis_2', 'arvalis_3', 'ethz_1', 'inrae_1', 'rres_1', 'usask_1'
\n\nThus, kRegionsOut validation strategies in training is useful to tackle test images from other regions.
\n\ncode snippet:\n> df = pd.read_csv('../input/global-wheat-detection/train.csv')\n> df.head()\n
Now mapping source to fold\n> mapping = {'arvalis_1': 0, 'arvalis_2': 1, 'arvalis_3': 2, 'ethz_1': 3, 'inrae_1': 4, 'rres_1': 5, 'usask_1': 6}
\n\n> df['folds'] = df.replace({'source': mapping}).source.values\ndf.head()\n
Number of unique images by source\n> df.groupby('source')['image_id'].nunique()\n
Number of unique images by fold\n> df.groupby('folds')['image_id'].nunique()\n
To get 'arvalis_1' region in validation set and all other regions in training:
\n\n\n", "rawMarkdown": "As @stavness mentioned in the [welcome post](https://www.kaggle.com/c/global-wheat-detection/discussion/148592), the test images are from different regions than training images:\n> blind test set of images from different world regions than in the training set\n\nWe find images from 7 different regions in training dataset.\n\n> 'arvalis\\_1', 'arvalis\\_2', 'arvalis\\_3', 'ethz\\_1', 'inrae\\_1', 'rres\\_1', 'usask\\_1'\n\nThus, **kRegionsOut** validation strategies in training is useful to tackle test images from other regions.\n\n**code snippet:**\n> df = pd.read_csv('../input/global-wheat-detection/train.csv')\n> df.head()\n\n\nNow mapping source to fold\n> mapping = {'arvalis\\_1': 0, 'arvalis\\_2': 1, 'arvalis\\_3': 2, 'ethz\\_1': 3, 'inrae\\_1': 4, 'rres\\_1': 5, 'usask\\_1': 6}\n\n> df['folds'] = df.replace({'source': mapping}).source.values\ndf.head()\n\n\nNumber of unique images by source\n> df.groupby('source')['image_id'].nunique()\n\n\nNumber of unique images by fold\n> df.groupby('folds')['image_id'].nunique()\n\n\nTo get 'arvalis\\_1' region in validation set and all other regions in training:\n\n> train\\_ids = df.loc[df.folds.values!=0,'image\\_id'].unique().tolist()\n> valid\\_ids = df.loc[df.folds.values==0,'image\\_id'].unique().tolist()\n> \n> train\\_df = df[df['image_id'].isin(train\\_ids)]\n\n> valid\\_df = df[df['image_id'].isin(valid\\_ids)]\n", "votes": 6 }, { "id": 834455, "postDate": "2020-05-05T14:48:42.620Z", "rawMarkdown": "", "votes": 1, "isDeleted": true, "replies": [ { "id": 834615, "postDate": "2020-05-05T16:47:52.227Z", "content": "train_ids = df.loc[df.folds.values!=0,'image_id'].unique().tolist()\n valid_ids = df.loc[df.folds.values==0,'image_id'].unique().tolist()
\n \ntrain_df = df[df['image_id'].isin(train_ids)]
\n \nvalid_df = df[df['image_id'].isin(valid_ids)]
\n
Yes, thank for pointing it. The main difficulty of this challenge is the bias in term of location. However the test images are also taken from nadir, with high resolution RGB images.
\n\nGood luck !
", "rawMarkdown": "Yes, thank for pointing it. The main difficulty of this challenge is the bias in term of location. However the test images are also taken from nadir, with high resolution RGB images.\n\nGood luck ! ", "votes": 2 } ] } ], "comments": [ { "id": 834455, "author_name": "", "author_url": "", "post_date": "2020-05-05T14:48:42.620000", "content": "", "votes": 1, "replies": [ { "id": 834615, "author_name": "Etienne David", "author_url": "", "post_date": "2020-05-05T16:47:52.227000", "content": "Yes, thank for pointing it. The main difficulty of this challenge is the bias in term of location. However the test images are also taken from nadir, with high resolution RGB images.
\n\nGood luck !
", "votes": 2, "replies": [] } ] } ], "raw_markdown_by_id": { "834102": "As @stavness mentioned in the [welcome post](https://www.kaggle.com/c/global-wheat-detection/discussion/148592), the test images are from different regions than training images:\n> blind test set of images from different world regions than in the training set\n\nWe find images from 7 different regions in training dataset.\n\n> 'arvalis\\_1', 'arvalis\\_2', 'arvalis\\_3', 'ethz\\_1', 'inrae\\_1', 'rres\\_1', 'usask\\_1'\n\nThus, **kRegionsOut** validation strategies in training is useful to tackle test images from other regions.\n\n**code snippet:**\n> df = pd.read_csv('../input/global-wheat-detection/train.csv')\n> df.head()\n\n\nNow mapping source to fold\n> mapping = {'arvalis\\_1': 0, 'arvalis\\_2': 1, 'arvalis\\_3': 2, 'ethz\\_1': 3, 'inrae\\_1': 4, 'rres\\_1': 5, 'usask\\_1': 6}\n\n> df['folds'] = df.replace({'source': mapping}).source.values\ndf.head()\n\n\nNumber of unique images by source\n> df.groupby('source')['image_id'].nunique()\n\n\nNumber of unique images by fold\n> df.groupby('folds')['image_id'].nunique()\n\n\nTo get 'arvalis\\_1' region in validation set and all other regions in training:\n\n> train\\_ids = df.loc[df.folds.values!=0,'image\\_id'].unique().tolist()\n> valid\\_ids = df.loc[df.folds.values==0,'image\\_id'].unique().tolist()\n> \n> train\\_df = df[df['image_id'].isin(train\\_ids)]\n\n> valid\\_df = df[df['image_id'].isin(valid\\_ids)]\n", "834455": "" } }