When one uploads a submission one sees this message:
\n\nNumber of Predictions\nWe expect the solution file to have 4152 prediction rows. This file should have a header row. Please see sample submission file on the data page.
\n\nWith 65 images that works out at as a average of 64 segmentations per image.\nIn the training file with 669 images, then as per stage1_train_labels.csv we have 29462 segmentations ( lines in the csv file) which averages at 44 segmentations per image.
\n\n44 in the training set, but an expectation of 64 in the test set ?
\n\nIt suggests they are not part of the same population. FD: In my solutions I can get to 43 segmentations per image in the test set, which lines up with the 44 of the training set, hence my question.
", "messages": [ { "id": "278720", "postDate": "02/06/2018 19:49:47", "content": "When one uploads a submission one sees this message:
\n\nNumber of Predictions\nWe expect the solution file to have 4152 prediction rows. This file should have a header row. Please see sample submission file on the data page.
\n\nWith 65 images that works out at as a average of 64 segmentations per image.\nIn the training file with 669 images, then as per stage1_train_labels.csv we have 29462 segmentations ( lines in the csv file) which averages at 44 segmentations per image.
\n\n44 in the training set, but an expectation of 64 in the test set ?
\n\nIt suggests they are not part of the same population. FD: In my solutions I can get to 43 segmentations per image in the test set, which lines up with the 44 of the training set, hence my question.
", "rawMarkdown": "When one uploads a submission one sees this message:\n\n *Number of Predictions\nWe expect the solution file to have 4152 prediction rows. This file should have a header row. Please see sample submission file on the data page.*\n\nWith 65 images that works out at as a average of 64 segmentations per image.\nIn the training file with 669 images, then as per stage1_train_labels.csv we have 29462 segmentations ( lines in the csv file) which averages at 44 segmentations per image.\n\n44 in the training set, but an expectation of 64 in the test set ?\n\nIt suggests they are not part of the same population. FD: In my solutions I can get to 43 segmentations per image in the test set, which lines up with the 44 of the training set, hence my question.", "votes": null }, { "id": "279128", "postDate": "02/07/2018 12:15:10", "content": "There are a number of outliers in the training set with > 100 masks. The median is actually 27.
\n\nIf you plot the images from the test set then you'll notice that many of them have a lot of nuclei, so it's not surprising that the average number there is greater than in the training set.
\n\nAlso, the goal of this competition is to create a model that generalizes well on other types of microscopy data. The current test set is therefore not completely similar to the training set, and in stage 2 the difference will probably be even larger.
", "rawMarkdown": "There are a number of outliers in the training set with > 100 masks. The median is actually 27.\n\nIf you plot the images from the test set then you'll notice that many of them have a lot of nuclei, so it's not surprising that the average number there is greater than in the training set.\n\nAlso, the goal of this competition is to create a model that generalizes well on other types of microscopy data. The current test set is therefore not completely similar to the training set, and in stage 2 the difference will probably be even larger.", "votes": null }, { "id": "279277", "postDate": "02/07/2018 17:43:38", "content": "Since this is a data science competition... a central assumption for using the test, train ( validation) methodology is that they are all randomly drawn from the same population( have the same probability distribution). As a fast check, I'll assume that the # of cells in an image follows the Poisson distribution. Then given 29462 segmentations in 669 images in training, 4152 in 65 images in test then comparing two poisson distributions to see if they have the same mean..
\n\npoisson.test(c(29462, 4152),r=1, c(669, 65), alternative = c(\"two.sided\")\n\n Comparison of Poisson rates\n\ndata: c(29462, 4152) time base: c(669, 65)\ncount1 = 29462, expected count1 = 30637, p-value < 2.2e-16\nalternative hypothesis: true rate ratio is not equal to 1\n95 percent confidence interval:\n0.6673682 0.7123694\nsample estimates:\nrate ratio \n 0.6894331 \nmeaning its unlikely the two are to have a ratio of lambda of 1 and unlikely to be randomly drawn from the same population. Or that its not a Poisson process. Still, its an interesting problem.
", "rawMarkdown": "Since this is a data science competition... *a central assumption for using the test, train ( validation) methodology is that they are all randomly drawn from the same population( have the same probability distribution)*. As a fast check, I'll assume that the # of cells in an image follows the Poisson distribution. Then given 29462 segmentations in 669 images in training, 4152 in 65 images in test then comparing two poisson distributions to see if they have the same mean..\n\n\tpoisson.test(c(29462, 4152),r=1, c(669, 65), alternative = c(\"two.sided\")\n\n\t\tComparison of Poisson rates\n\n\tdata: c(29462, 4152) time base: c(669, 65)\n\tcount1 = 29462, expected count1 = 30637, p-value < 2.2e-16\n\talternative hypothesis: true rate ratio is not equal to 1\n\t95 percent confidence interval:\n \t0.6673682 0.7123694\n\tsample estimates:\n\trate ratio \n\t 0.6894331 \n\nmeaning its unlikely the two are to have a ratio of lambda of 1 and unlikely to be randomly drawn from the same population. Or that its not a Poisson process. Still, its an interesting problem.", "votes": null }, { "id": "279279", "postDate": "02/07/2018 17:48:54", "content": "dup - deleted..
", "rawMarkdown": "dup - deleted..", "votes": null }, { "id": "289979", "postDate": "03/02/2018 18:12:26", "content": "Hi @skthetwo, thanks for reporting this. We deployed a fix yesterday and this message shouldn't be showing anymore. Please let me know if there are other problems.
", "rawMarkdown": "Hi @skthetwo, thanks for reporting this. We deployed a fix yesterday and this message shouldn't be showing anymore. Please let me know if there are other problems.", "votes": null }, { "id": "290384", "postDate": "03/03/2018 23:14:18", "content": "The closer to 4152 the number of rows in my sub, the higher the LB score (so far).
", "rawMarkdown": "The closer to 4152 the number of rows in my sub, the higher the LB score (so far).", "votes": null } ], "comments": [ { "id": 279128, "author_name": "humananalog", "author_url": "", "post_date": "02/07/2018 12:15:10", "content": "There are a number of outliers in the training set with > 100 masks. The median is actually 27.
\n\nIf you plot the images from the test set then you'll notice that many of them have a lot of nuclei, so it's not surprising that the average number there is greater than in the training set.
\n\nAlso, the goal of this competition is to create a model that generalizes well on other types of microscopy data. The current test set is therefore not completely similar to the training set, and in stage 2 the difference will probably be even larger.
", "votes": null, "replies": [] }, { "id": 279277, "author_name": "skthetwo", "author_url": "", "post_date": "02/07/2018 17:43:38", "content": "Since this is a data science competition... a central assumption for using the test, train ( validation) methodology is that they are all randomly drawn from the same population( have the same probability distribution). As a fast check, I'll assume that the # of cells in an image follows the Poisson distribution. Then given 29462 segmentations in 669 images in training, 4152 in 65 images in test then comparing two poisson distributions to see if they have the same mean..
\n\npoisson.test(c(29462, 4152),r=1, c(669, 65), alternative = c(\"two.sided\")\n\n Comparison of Poisson rates\n\ndata: c(29462, 4152) time base: c(669, 65)\ncount1 = 29462, expected count1 = 30637, p-value < 2.2e-16\nalternative hypothesis: true rate ratio is not equal to 1\n95 percent confidence interval:\n0.6673682 0.7123694\nsample estimates:\nrate ratio \n 0.6894331 \nmeaning its unlikely the two are to have a ratio of lambda of 1 and unlikely to be randomly drawn from the same population. Or that its not a Poisson process. Still, its an interesting problem.
", "votes": null, "replies": [] }, { "id": 279279, "author_name": "skthetwo", "author_url": "", "post_date": "02/07/2018 17:48:54", "content": "dup - deleted..
", "votes": null, "replies": [] }, { "id": 289979, "author_name": "wendykan", "author_url": "", "post_date": "03/02/2018 18:12:26", "content": "Hi @skthetwo, thanks for reporting this. We deployed a fix yesterday and this message shouldn't be showing anymore. Please let me know if there are other problems.
", "votes": null, "replies": [] }, { "id": 290384, "author_name": "cpmpml", "author_url": "", "post_date": "03/03/2018 23:14:18", "content": "The closer to 4152 the number of rows in my sub, the higher the LB score (so far).
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "278720": "When one uploads a submission one sees this message:\n\n *Number of Predictions\nWe expect the solution file to have 4152 prediction rows. This file should have a header row. Please see sample submission file on the data page.*\n\nWith 65 images that works out at as a average of 64 segmentations per image.\nIn the training file with 669 images, then as per stage1_train_labels.csv we have 29462 segmentations ( lines in the csv file) which averages at 44 segmentations per image.\n\n44 in the training set, but an expectation of 64 in the test set ?\n\nIt suggests they are not part of the same population. FD: In my solutions I can get to 43 segmentations per image in the test set, which lines up with the 44 of the training set, hence my question.", "279128": "There are a number of outliers in the training set with > 100 masks. The median is actually 27.\n\nIf you plot the images from the test set then you'll notice that many of them have a lot of nuclei, so it's not surprising that the average number there is greater than in the training set.\n\nAlso, the goal of this competition is to create a model that generalizes well on other types of microscopy data. The current test set is therefore not completely similar to the training set, and in stage 2 the difference will probably be even larger.", "279277": "Since this is a data science competition... *a central assumption for using the test, train ( validation) methodology is that they are all randomly drawn from the same population( have the same probability distribution)*. As a fast check, I'll assume that the # of cells in an image follows the Poisson distribution. Then given 29462 segmentations in 669 images in training, 4152 in 65 images in test then comparing two poisson distributions to see if they have the same mean..\n\n\tpoisson.test(c(29462, 4152),r=1, c(669, 65), alternative = c(\"two.sided\")\n\n\t\tComparison of Poisson rates\n\n\tdata: c(29462, 4152) time base: c(669, 65)\n\tcount1 = 29462, expected count1 = 30637, p-value < 2.2e-16\n\talternative hypothesis: true rate ratio is not equal to 1\n\t95 percent confidence interval:\n \t0.6673682 0.7123694\n\tsample estimates:\n\trate ratio \n\t 0.6894331 \n\nmeaning its unlikely the two are to have a ratio of lambda of 1 and unlikely to be randomly drawn from the same population. Or that its not a Poisson process. Still, its an interesting problem.", "279279": "dup - deleted..", "289979": "Hi @skthetwo, thanks for reporting this. We deployed a fix yesterday and this message shouldn't be showing anymore. Please let me know if there are other problems.", "290384": "The closer to 4152 the number of rows in my sub, the higher the LB score (so far)." }, "source": "meta" }