Today we had our first \r\nweekly teleconference seminar in which you have an opportunity to ask questions to the experts (held every Thursday at 9am PT).
\r\nThe discussion included the following topics:", "messages": [ { "id": "24112", "postDate": "05/09/2013 17:52:08", "content": "
1 - final evaluation: artificial vs real data
2 - how to detect artifacts in artificial data
3 - release of intermediate finer analysis (real vs artificial, all numerical vs mixed with categorical/binary)
4 - how to generate real data for the case A-B
5 - running and test time requirements
--
1) Should the final evaluation include only real data or both artificial and real data? The participants were favorable to include both because there is as much risk to introduce artifacts in real data as there is in artificial data while collecting and preparing the data. The organizers proposed a possible change in prize attribution: having a prize for best result overall, one for best on artificial data and one for best on real data, instead of 3 prizes for the combined score.
2) How can the participants find out about artifacts in data? The can conduct a sensitivity analysis on features used. If one feature removed yields significant performance degradation and that feature "does not make sense", then there is a possible problem with the data. How can you avoid risking giving away your best features by disclosing this? Contact the organizers at causality at chalearn dot org. They will disclose this information only if there is a problem identified with the data in connection to the way it was generated.
3) A participant requested that a more detailed milestone analysis about the results on validation data be disclosed, including results on real vs artificial data and results on all numerical pairs vs mixed pairs.
4) A participant asked how could one find out whether a pair {A, B} of real data is of the type A - B. Assume that you know that a given variable is a common cause of several others. For example, gene expression analysis of leukemia patients submitted to a radiation treatment: the treatment is the cause, the gene expressions are the consequences. A pair A-B can be obtained from 2 genes that are independent from each other given the treatment.
5) Are there limitations in the training and test time? For training time, there are no limitations as long as you produce a trained model for the final evaluation before the deadline for software submission Monday, July 15, 2013. For test time, you need to be able to process the test data between July 19 and July 26 (one week). The test data will include ~10,000 pairs (depending on how much data are donated by the participants).
Please join the next discussion Thursday, May 16, 2013, 9 am PT. The speaker Dominik Janzing is one of the foremost expert in the field and will be talking about information Geometry methods to detecting cause-effect relationships.
Conference Dial-in Number: +1 (610) 214-0000
Participant Access Code: 549086#
Today we had our first \r\nweekly teleconference seminar in which you have an opportunity to ask questions to the experts (held every Thursday at 9am PT).
\r\nThe discussion included the following topics:", "rawMarkdown": "", "votes": null }, { "id": "24347", "postDate": "05/15/2013 14:44:25", "content": "
1 - final evaluation: artificial vs real data
2 - how to detect artifacts in artificial data
3 - release of intermediate finer analysis (real vs artificial, all numerical vs mixed with categorical/binary)
4 - how to generate real data for the case A-B
5 - running and test time requirements
--
1) Should the final evaluation include only real data or both artificial and real data? The participants were favorable to include both because there is as much risk to introduce artifacts in real data as there is in artificial data while collecting and preparing the data. The organizers proposed a possible change in prize attribution: having a prize for best result overall, one for best on artificial data and one for best on real data, instead of 3 prizes for the combined score.
2) How can the participants find out about artifacts in data? The can conduct a sensitivity analysis on features used. If one feature removed yields significant performance degradation and that feature "does not make sense", then there is a possible problem with the data. How can you avoid risking giving away your best features by disclosing this? Contact the organizers at causality at chalearn dot org. They will disclose this information only if there is a problem identified with the data in connection to the way it was generated.
3) A participant requested that a more detailed milestone analysis about the results on validation data be disclosed, including results on real vs artificial data and results on all numerical pairs vs mixed pairs.
4) A participant asked how could one find out whether a pair {A, B} of real data is of the type A - B. Assume that you know that a given variable is a common cause of several others. For example, gene expression analysis of leukemia patients submitted to a radiation treatment: the treatment is the cause, the gene expressions are the consequences. A pair A-B can be obtained from 2 genes that are independent from each other given the treatment.
5) Are there limitations in the training and test time? For training time, there are no limitations as long as you produce a trained model for the final evaluation before the deadline for software submission Monday, July 15, 2013. For test time, you need to be able to process the test data between July 19 and July 26 (one week). The test data will include ~10,000 pairs (depending on how much data are donated by the participants).
Please join the next discussion Thursday, May 16, 2013, 9 am PT. The speaker Dominik Janzing is one of the foremost expert in the field and will be talking about information Geometry methods to detecting cause-effect relationships.
Conference Dial-in Number: +1 (610) 214-0000
Participant Access Code: 549086#
Reminder:
\r\n| Thursday, May 16, 2013 | \r\n9 am PT | \r\nDominik Janzing | \r\nInformation Geometry method to detecting cause-effect relationships | \r\nView Download | \r\n\r\n |
Exceptionally this week, the reading group is meeting wednesday (tomorrow) instead of thursday:
\r\nhttps://sites.google.com/a/chalearn.org/readinggroup/home
| Wednesday, June 5, 2013 | \r\n9 am PT | \r\nJoris Mooij | \r\nMML methods for detecting cause-effect relationships | \r\nPaper View\r\n slides Slides(pdf) | \r\n\r\n |
Reminder:
\r\n| Thursday, June 13, 2013 | \r\n9 am PT | \r\nKun Zhang | \r\nIdentifying cause-effect relationships with non-linear functional models. | \r\nView Download | \r\n\r\n |
Reminder:
\r\n| Thursday, May 16, 2013 | \r\n9 am PT | \r\nDominik Janzing | \r\nInformation Geometry method to detecting cause-effect relationships | \r\nView Download | \r\n\r\n |
Exceptionally this week, the reading group is meeting wednesday (tomorrow) instead of thursday:
\r\nhttps://sites.google.com/a/chalearn.org/readinggroup/home
| Wednesday, June 5, 2013 | \r\n9 am PT | \r\nJoris Mooij | \r\nMML methods for detecting cause-effect relationships | \r\nPaper View\r\n slides Slides(pdf) | \r\n\r\n |
Reminder:
\r\n| Thursday, June 13, 2013 | \r\n9 am PT | \r\nKun Zhang | \r\nIdentifying cause-effect relationships with non-linear functional models. | \r\nView Download | \r\n\r\n |