There are duplicates in molecule names in several training dataset (ACT1, 6, 8, 10). But variable values of those molecules are different. Also there are a lot of duplicates in names of variables in training and test sets. But values of variables with identical\r\n names are different. (I didn't check all datasets comprehensively yet).
\r\nWhat does it mean? Are they mistakes or not?
", "messages": [ { "id": "13188", "postDate": "08/18/2012 14:19:19", "content": "There are duplicates in molecule names in several training dataset (ACT1, 6, 8, 10). But variable values of those molecules are different. Also there are a lot of duplicates in names of variables in training and test sets. But values of variables with identical\r\n names are different. (I didn't check all datasets comprehensively yet).
\r\nWhat does it mean? Are they mistakes or not?
", "rawMarkdown": "", "votes": null }, { "id": "13189", "postDate": "08/18/2012 14:31:13", "content": "As described on the \r\ndata page, some molecules and some descriptors appear in multiple data sets. That means a given molecule was tested in multiple panels/assays to get descriptors and may have multiple types of activity.
", "rawMarkdown": "", "votes": null }, { "id": "13190", "postDate": "08/18/2012 16:06:54", "content": "I apologize for ambiguity in my previous post. I mean that there are variables with identical names within each separate dataset. And the same for molecules. This isn't described in data description.
", "rawMarkdown": "", "votes": null }, { "id": "13198", "postDate": "08/18/2012 21:28:54", "content": "Drrr, thanks for the clarification and finding the error. You were correct. I've fixed the changes and updated the TrainingSet file. Please download the new files.
", "rawMarkdown": "", "votes": null }, { "id": "13211", "postDate": "08/19/2012 05:00:22", "content": "Thanks.
\r\nI cannot check all datasets again, but I just look at one dataset (ACT10) and found that variables with identical names are still present. Please pay attantion, that there a lot of variables with identical names and probably different values in all training\r\n and test sets. For example in ACT10 dataset the variable DOkX8OkX88 is repeated twice, etc.
\r\nIf you need I can send you names of all duplicated variables in each dataset. But it can be easily retrived by colnames(df)[duplicated(colnames(df))]. Check please.
Hi Drrr, I've checked all of the data sets and there are no duplicated column names (descriptor names) in each data set for any of the data sets.
\r\nI think you might be changing the headers when reading the file. For instance, you pointed to descriptor = "DOkX8OkX88", but this string does not exist in ACT10. The actual descriptor names in ACT10 are "D_OkX8OkX88_" and "D_OkX8OkX88_.1" (cols 4, 5). These\r\n are unique strings and thus unique column ids. This happens again with cols 7, 8 with "D_OkX8OkX8X_" and "D_OkX8OkX8X_.1" and actually quite often throughout, which might be why you think this is happening repeatedly.
", "rawMarkdown": "", "votes": null }, { "id": "13222", "postDate": "08/19/2012 17:53:57", "content": "Yes, if you use default options during loading dataset in R (check.names=TRUE), all duplicated column names will be changed in the manner you desribed. But in initial dataset there a lot of duplicated variable names, please look at csv-files. That is why\r\n I never use defualt options at file loading to avoid implicit conversions.
\r\ndf <- read.table(file.name, header=T, sep=",", check.names=F, as.is=T)
\r\ncolnames(df)[duplicated(colnames(df))]
[quote=jcnhvnhck;13220]
\r\nFor instance, you pointed to descriptor = "DOkX8OkX88", but this string does not exist in ACT10. The actual descriptor names in ACT10 are "D_OkX8OkX88_" and "D_OkX8OkX88_.1" (cols 4, 5). These are unique strings and thus unique column ids.
\r\n[/quote]
\r\nThere is no "D_OkX8OkX8X_.1" in the downloaded ACT10_competition_training.csv (in TrainingSet.zip).
\r\nThe header starts with "MOLECULE,Act,D_OkX8OkX88Y,D_OkX8OkX88_,D_OkX8OkX88_,D_OkX8OkX8XX,D_OkX8OkX8X_,"
\r\n
\r\n
Hey all,
\r\nthere are like 9804 duplicate attribute names in both train and test files (some attributes appear 3,4,5 or 6 times). Check added file for them if You are interested.
\r\n1st col: filename
\r\n2nd col: colname
\r\n3rd col: dupe count
\r\n4th col: indexes of dupe in the list of the header of the file
\r\nBest,
\r\nivo
", "rawMarkdown": "", "votes": null }, { "id": "13233", "postDate": "08/20/2012 13:13:38", "content": "Drrr and ivo, Thanks to you both for your help with this. I understand the problem and am working with the Competition Sponsor to get the corrected data sets. I will post a new forum post when the new files are ready later today.
", "rawMarkdown": "", "votes": null }, { "id": "13298", "postDate": "08/22/2012 00:05:02", "content": "We are still waiting for new files right? (kind of anxious to have them :) )
", "rawMarkdown": "", "votes": null }, { "id": "13299", "postDate": "08/22/2012 00:22:43", "content": "Some variables such as DPX8PX88.1 in set 2 have very different distributions in the training set and the testing set.\r\n
\r\n\r\n", "rawMarkdown": "", "votes": null }, { "id": "13300", "postDate": "08/22/2012 01:23:23", "content": "table(training.E2$DPX8PX88.1)
\r\n
\r\n0 1 2 3 4 5 6 8
\r\n6586 237 1720 96 59 2 14 2
\r\ntable(test.E2$DPX8PX88.1)
\r\n0 1 2 3 9
\r\n2550 106 232 18 1
\r\nI guess this issue will be fixed with the new files.
\r\nThanks, Xavier
Thank you everyone for your patience as we worked through the data issues. New data files are now\r\navailable. Molecule names and descriptors names have been simplified to avoid special characters and avoid file reading problems. Also, files should contain only unique molecules and unique descriptors\r\n within each data set. As described previously, some molecules and descriptors will appear in multiple data sets.
", "rawMarkdown": "", "votes": null }, { "id": "13316", "postDate": "08/22/2012 10:34:52", "content": "[quote=Gxav (Xavier Conort);13299]
\r\nSome variables such as DPX8PX88.1 in set 2 have very different distributions in the training set and the testing set.
\r\n\r\n\r\ntable(training.E2$DPX8PX88.1)
\r\n
\r\n0 1 2 3 4 5 6 8
\r\n6586 237 1720 96 59 2 14 2
\r\ntable(test.E2$DPX8PX88.1)
\r\n0 1 2 3 9
\r\n2550 106 232 18 1
\r\nI guess this issue will be fixed with the new files.
\r\nThanks, Xavier
[/quote]
\r\nHow the test molecules are selected? randomly or test set are an specific molecules type that the promoter has interest in them?
\r\nThe test set generation must be detailed in each challenge profile.
", "rawMarkdown": "", "votes": null }, { "id": "13329", "postDate": "08/22/2012 13:04:39", "content": "Re: Test Set Generation
\r\nIs it really that hard to figure out if the test and training sets are random samples from the same population? I realize it's fairly high dimensional data, but you can reduce those dimensions and make visual comparisons.
\r\nI don't recall reading anything that says they have to confirm how they made the test set. Furthermore, I don't think I'd trust anything that was said since there are many cooks in the kitchen (Kaggle & Merck at least). It's not that hard to run some tests\r\n yourself.
", "rawMarkdown": "", "votes": null }, { "id": "13388", "postDate": "08/24/2012 03:43:30", "content": "I finally spent some time with this data.
\r\nGiven the current test & train sets, I do not believe they were sampled form the same population at all. If you graph the points by their first couple principal components, you can see clear separation between test & train data.
\r\nWith this knowledge, I would not expect standard cross-validated error estimates to give anything approaching a leaderboard score.
", "rawMarkdown": "", "votes": null }, { "id": "13405", "postDate": "08/24/2012 15:14:35", "content": "Because who doesn't enjoy pretty pictures? Here's probably the most extreme example of separation I found (Activity 5). Red=Test, Blue=Train You'll have to take it on faith I did the dimensionality reduction correctly. Well, that or you can do it yourself\r\n to confirm.", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 13189, "author_name": "joycenv", "author_url": "", "post_date": "08/18/2012 14:31:13", "content": "As described on the \r\ndata page, some molecules and some descriptors appear in multiple data sets. That means a given molecule was tested in multiple panels/assays to get descriptors and may have multiple types of activity.
", "votes": null, "replies": [] }, { "id": 13190, "author_name": "drrr11321", "author_url": "", "post_date": "08/18/2012 16:06:54", "content": "I apologize for ambiguity in my previous post. I mean that there are variables with identical names within each separate dataset. And the same for molecules. This isn't described in data description.
", "votes": null, "replies": [] }, { "id": 13198, "author_name": "joycenv", "author_url": "", "post_date": "08/18/2012 21:28:54", "content": "Drrr, thanks for the clarification and finding the error. You were correct. I've fixed the changes and updated the TrainingSet file. Please download the new files.
", "votes": null, "replies": [] }, { "id": 13211, "author_name": "drrr11321", "author_url": "", "post_date": "08/19/2012 05:00:22", "content": "Thanks.
\r\nI cannot check all datasets again, but I just look at one dataset (ACT10) and found that variables with identical names are still present. Please pay attantion, that there a lot of variables with identical names and probably different values in all training\r\n and test sets. For example in ACT10 dataset the variable DOkX8OkX88 is repeated twice, etc.
\r\nIf you need I can send you names of all duplicated variables in each dataset. But it can be easily retrived by colnames(df)[duplicated(colnames(df))]. Check please.
Hi Drrr, I've checked all of the data sets and there are no duplicated column names (descriptor names) in each data set for any of the data sets.
\r\nI think you might be changing the headers when reading the file. For instance, you pointed to descriptor = "DOkX8OkX88", but this string does not exist in ACT10. The actual descriptor names in ACT10 are "D_OkX8OkX88_" and "D_OkX8OkX88_.1" (cols 4, 5). These\r\n are unique strings and thus unique column ids. This happens again with cols 7, 8 with "D_OkX8OkX8X_" and "D_OkX8OkX8X_.1" and actually quite often throughout, which might be why you think this is happening repeatedly.
", "votes": null, "replies": [] }, { "id": 13222, "author_name": "drrr11321", "author_url": "", "post_date": "08/19/2012 17:53:57", "content": "Yes, if you use default options during loading dataset in R (check.names=TRUE), all duplicated column names will be changed in the manner you desribed. But in initial dataset there a lot of duplicated variable names, please look at csv-files. That is why\r\n I never use defualt options at file loading to avoid implicit conversions.
\r\ndf <- read.table(file.name, header=T, sep=",", check.names=F, as.is=T)
\r\ncolnames(df)[duplicated(colnames(df))]
[quote=jcnhvnhck;13220]
\r\nFor instance, you pointed to descriptor = "DOkX8OkX88", but this string does not exist in ACT10. The actual descriptor names in ACT10 are "D_OkX8OkX88_" and "D_OkX8OkX88_.1" (cols 4, 5). These are unique strings and thus unique column ids.
\r\n[/quote]
\r\nThere is no "D_OkX8OkX8X_.1" in the downloaded ACT10_competition_training.csv (in TrainingSet.zip).
\r\nThe header starts with "MOLECULE,Act,D_OkX8OkX88Y,D_OkX8OkX88_,D_OkX8OkX88_,D_OkX8OkX8XX,D_OkX8OkX8X_,"
\r\n
\r\n
Hey all,
\r\nthere are like 9804 duplicate attribute names in both train and test files (some attributes appear 3,4,5 or 6 times). Check added file for them if You are interested.
\r\n1st col: filename
\r\n2nd col: colname
\r\n3rd col: dupe count
\r\n4th col: indexes of dupe in the list of the header of the file
\r\nBest,
\r\nivo
", "votes": null, "replies": [] }, { "id": 13233, "author_name": "joycenv", "author_url": "", "post_date": "08/20/2012 13:13:38", "content": "Drrr and ivo, Thanks to you both for your help with this. I understand the problem and am working with the Competition Sponsor to get the corrected data sets. I will post a new forum post when the new files are ready later today.
", "votes": null, "replies": [] }, { "id": 13298, "author_name": "jscheibel", "author_url": "", "post_date": "08/22/2012 00:05:02", "content": "We are still waiting for new files right? (kind of anxious to have them :) )
", "votes": null, "replies": [] }, { "id": 13299, "author_name": "xavierconort", "author_url": "", "post_date": "08/22/2012 00:22:43", "content": "Some variables such as DPX8PX88.1 in set 2 have very different distributions in the training set and the testing set.\r\n
\r\n\r\n", "votes": null, "replies": [] }, { "id": 13300, "author_name": "joycenv", "author_url": "", "post_date": "08/22/2012 01:23:23", "content": "table(training.E2$DPX8PX88.1)
\r\n
\r\n0 1 2 3 4 5 6 8
\r\n6586 237 1720 96 59 2 14 2
\r\ntable(test.E2$DPX8PX88.1)
\r\n0 1 2 3 9
\r\n2550 106 232 18 1
\r\nI guess this issue will be fixed with the new files.
\r\nThanks, Xavier
Thank you everyone for your patience as we worked through the data issues. New data files are now\r\navailable. Molecule names and descriptors names have been simplified to avoid special characters and avoid file reading problems. Also, files should contain only unique molecules and unique descriptors\r\n within each data set. As described previously, some molecules and descriptors will appear in multiple data sets.
", "votes": null, "replies": [] }, { "id": 13316, "author_name": "blindape", "author_url": "", "post_date": "08/22/2012 10:34:52", "content": "[quote=Gxav (Xavier Conort);13299]
\r\nSome variables such as DPX8PX88.1 in set 2 have very different distributions in the training set and the testing set.
\r\n\r\n\r\ntable(training.E2$DPX8PX88.1)
\r\n
\r\n0 1 2 3 4 5 6 8
\r\n6586 237 1720 96 59 2 14 2
\r\ntable(test.E2$DPX8PX88.1)
\r\n0 1 2 3 9
\r\n2550 106 232 18 1
\r\nI guess this issue will be fixed with the new files.
\r\nThanks, Xavier
[/quote]
\r\nHow the test molecules are selected? randomly or test set are an specific molecules type that the promoter has interest in them?
\r\nThe test set generation must be detailed in each challenge profile.
", "votes": null, "replies": [] }, { "id": 13329, "author_name": "biasvariance", "author_url": "", "post_date": "08/22/2012 13:04:39", "content": "Re: Test Set Generation
\r\nIs it really that hard to figure out if the test and training sets are random samples from the same population? I realize it's fairly high dimensional data, but you can reduce those dimensions and make visual comparisons.
\r\nI don't recall reading anything that says they have to confirm how they made the test set. Furthermore, I don't think I'd trust anything that was said since there are many cooks in the kitchen (Kaggle & Merck at least). It's not that hard to run some tests\r\n yourself.
", "votes": null, "replies": [] }, { "id": 13388, "author_name": "biasvariance", "author_url": "", "post_date": "08/24/2012 03:43:30", "content": "I finally spent some time with this data.
\r\nGiven the current test & train sets, I do not believe they were sampled form the same population at all. If you graph the points by their first couple principal components, you can see clear separation between test & train data.
\r\nWith this knowledge, I would not expect standard cross-validated error estimates to give anything approaching a leaderboard score.
", "votes": null, "replies": [] }, { "id": 13405, "author_name": "biasvariance", "author_url": "", "post_date": "08/24/2012 15:14:35", "content": "Because who doesn't enjoy pretty pictures? Here's probably the most extreme example of separation I found (Activity 5). Red=Test, Blue=Train You'll have to take it on faith I did the dimensionality reduction correctly. Well, that or you can do it yourself\r\n to confirm.", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "13188": "", "13189": "", "13190": "", "13198": "", "13211": "", "13220": "", "13222": "", "13223": "", "13224": "", "13233": "", "13298": "", "13299": "", "13300": "", "13316": "", "13329": "", "13388": "", "13405": "" }, "source": "meta" }