![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1314380%2F86f4e6e6df673ac81a80abbfdc31331d%2FScreen%20Shot%202020-10-27%20at%204.05.58%20PM.png?generation=1603836372617867&alt=media\")
Hi, I am having a go at this competition.
\nI noticed that possibly some data is missing. For example for the 2020 Corporate questionnaire only has answers to the question C1.1a . That should be not possible as that question is accessible only if C1.1 is answered. Additionally other questions from the Governance section of the questionnaire appear to be missing (as an example). Is it possible the data has been truncated or am I doing something wrong?
Many thanks.
", "messages": [ { "id": "1062478", "postDate": "10/27/2020 21:50:53", "content": "Hi, I am having a go at this competition.
\nI noticed that possibly some data is missing. For example for the 2020 Corporate questionnaire only has answers to the question C1.1a . That should be not possible as that question is accessible only if C1.1 is answered. Additionally other questions from the Governance section of the questionnaire appear to be missing (as an example). Is it possible the data has been truncated or am I doing something wrong?
Many thanks.
", "rawMarkdown": "Hi, I am having a go at this competition. \nI noticed that possibly some data is missing. For example for the 2020 Corporate questionnaire only has answers to the question C1.1a . That should be not possible as that question is accessible only if C1.1 is answered. Additionally other questions from the Governance section of the questionnaire appear to be missing (as an example). Is it possible the data has been truncated or am I doing something wrong?\n\nMany thanks.", "votes": null }, { "id": "1062491", "postDate": "10/27/2020 22:10:38", "content": "The main sources of data should be the .CSV files.
\n
The CSV files have much more data than just C1.1:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1314380%2F1507b88fb480c4a6a47c02e538e415ff%2FScreen%20Shot%202020-10-27%20at%204.08.16%20PM.png?generation=1603836542200013&alt=media\")
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1314380%2Fca0a821d7af2ef793d30975996628e94%2FScreen%20Shot%202020-10-27%20at%204.08.52%20PM.png?generation=1603836559667640&alt=media\")
Likewise, here is the governance section from both of the 2020 questionnaires:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1314380%2F4a7fda487747f3cc55a972a70ae51fcb%2FScreen%20Shot%202020-10-27%20at%204.04.06%20PM.png?generation=1603836299382161&alt=media\")
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1314380%2F9c37fb338a4d4eae04a0f31d1bed90bc%2FScreen%20Shot%202020-10-27%20at%204.04.13%20PM.png?generation=1603836307922797&alt=media\")
Am I misunderstanding your question?
", "rawMarkdown": "The main sources of [data](https://www.kaggle.com/c/cdp-unlocking-climate-solutions/data) should be the .CSV files. \n\n\n\nThe CSV files have much more data than just C1.1:\n\n\n\n\n\nLikewise, here is the governance section from both of the 2020 questionnaires:\n\n\n\n\n\nAm I misunderstanding your question?", "votes": null }, { "id": "1062513", "postDate": "10/27/2020 23:14:22", "content": "What I meant is shouldn't there be a question number C1.1, C1.1b c1.1c as in the pdf? When I look at data I see only C1.1a it seems that those other questions that are present in the pdf \"2020_Climate_Change_Questionnaire\" but the data only contains answers to question C1.1a. Try run the following and compare with the pdf questionnaire structure for the governance section:
\nimport pandas as pd
\nq2020_crop = pd.read_csv('../input/cdp-unlocking-climate-solutions/Corporations/Corporations Responses/Climate Change/2020_Full_Climate_Change_Dataset.csv')
\npd.options.display.max_rows = 1000
\nq2020_crop.groupby([\"question_number\"]).size()
Or better, use this code:
\nimport pandas as pd
\nq2020_crop = pd.read_csv('../input/cdp-unlocking-climate-solutions/Corporations/Corporations Responses/Climate Change/2020_Full_Climate_Change_Dataset.csv')
\npd.options.display.max_rows = 1000
\nq2020_crop.groupby([\"module_name\",\"question_number\"]).size()
The file on cities is correct with respect to the pdf questionnaire.
\nBut for corporations, some modules are missing when comparing the pdf and csv files
Thanks for the questions!
\nDue to compliance and legal restrictions, we are only able to include a limited amount of question responses within the corporate data sets available to you in the .csvs.
\nSo although you can see ALL the questions that company's are presented with in the PDFs, you are only provided with question responses that we are legally permitted to share publicly within the .csvs.
\ne.g. that's why you can only find responses to C1.1a within the dataset - we are not permitted to publicly release the responses to to C1.1, C1.1b or C1.1c within the context of this competitions dataset
\nThe PDFs are there to give you more contextual knowledge around how our questions are designed, but include questions that you do not have response data for.
\nHope this clears things up!
", "rawMarkdown": "Thanks for the questions!\n\nDue to compliance and legal restrictions, we are only able to include a limited amount of question responses within the corporate data sets available to you in the .csvs. \n\nSo although you can see ALL the questions that company's are presented with in the PDFs, you are only provided with question responses that we are legally permitted to share publicly within the .csvs.\n\n*e.g. that's why you can only find responses to C1.1a within the dataset - we are not permitted to publicly release the responses to to C1.1, C1.1b or C1.1c within the context of this competitions dataset*\n\nThe PDFs are there to give you more contextual knowledge around how our questions are designed, but include questions that you do not have response data for.\n\nHope this clears things up!", "votes": null }, { "id": "1063343", "postDate": "10/28/2020 18:59:37", "content": "As mentioned by Callum above (just recently), a subset of data points from the corporate disclosures have been redacted to ensure legal compliance.
", "rawMarkdown": "As mentioned by Callum [above](https://www.kaggle.com/c/cdp-unlocking-climate-solutions/discussion/193612#1063309) (just recently), a subset of data points from the corporate disclosures have been redacted to ensure legal compliance.", "votes": null }, { "id": "1063384", "postDate": "10/28/2020 20:18:04", "content": "Many thanks for your prompt reply. This is really helpful.
", "rawMarkdown": "Many thanks for your prompt reply. This is really helpful.", "votes": null } ], "comments": [ { "id": 1062491, "author_name": "paultimothymooney", "author_url": "", "post_date": "10/27/2020 22:10:38", "content": "The main sources of data should be the .CSV files.
\n
The CSV files have much more data than just C1.1:
\n
Likewise, here is the governance section from both of the 2020 questionnaires:
\n
Am I misunderstanding your question?
", "votes": null, "replies": [ { "id": 1063159, "author_name": "jbrans", "author_url": "", "post_date": "10/28/2020 15:10:59", "content": "The file on cities is correct with respect to the pdf questionnaire.
\nBut for corporations, some modules are missing when comparing the pdf and csv files
As mentioned by Callum above (just recently), a subset of data points from the corporate disclosures have been redacted to ensure legal compliance.
", "votes": null, "replies": [] } ] }, { "id": 1062513, "author_name": "david289", "author_url": "", "post_date": "10/27/2020 23:14:22", "content": "What I meant is shouldn't there be a question number C1.1, C1.1b c1.1c as in the pdf? When I look at data I see only C1.1a it seems that those other questions that are present in the pdf \"2020_Climate_Change_Questionnaire\" but the data only contains answers to question C1.1a. Try run the following and compare with the pdf questionnaire structure for the governance section:
\nimport pandas as pd
\nq2020_crop = pd.read_csv('../input/cdp-unlocking-climate-solutions/Corporations/Corporations Responses/Climate Change/2020_Full_Climate_Change_Dataset.csv')
\npd.options.display.max_rows = 1000
\nq2020_crop.groupby([\"question_number\"]).size()
Or better, use this code:
\nimport pandas as pd
\nq2020_crop = pd.read_csv('../input/cdp-unlocking-climate-solutions/Corporations/Corporations Responses/Climate Change/2020_Full_Climate_Change_Dataset.csv')
\npd.options.display.max_rows = 1000
\nq2020_crop.groupby([\"module_name\",\"question_number\"]).size()
Thanks for the questions!
\nDue to compliance and legal restrictions, we are only able to include a limited amount of question responses within the corporate data sets available to you in the .csvs.
\nSo although you can see ALL the questions that company's are presented with in the PDFs, you are only provided with question responses that we are legally permitted to share publicly within the .csvs.
\ne.g. that's why you can only find responses to C1.1a within the dataset - we are not permitted to publicly release the responses to to C1.1, C1.1b or C1.1c within the context of this competitions dataset
\nThe PDFs are there to give you more contextual knowledge around how our questions are designed, but include questions that you do not have response data for.
\nHope this clears things up!
", "votes": null, "replies": [ { "id": 1063384, "author_name": "david289", "author_url": "", "post_date": "10/28/2020 20:18:04", "content": "Many thanks for your prompt reply. This is really helpful.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1062478": "Hi, I am having a go at this competition. \nI noticed that possibly some data is missing. For example for the 2020 Corporate questionnaire only has answers to the question C1.1a . That should be not possible as that question is accessible only if C1.1 is answered. Additionally other questions from the Governance section of the questionnaire appear to be missing (as an example). Is it possible the data has been truncated or am I doing something wrong?\n\nMany thanks.", "1062491": "The main sources of [data](https://www.kaggle.com/c/cdp-unlocking-climate-solutions/data) should be the .CSV files. \n\n\n\nThe CSV files have much more data than just C1.1:\n\n\n\n\n\nLikewise, here is the governance section from both of the 2020 questionnaires:\n\n\n\n\n\nAm I misunderstanding your question?", "1062513": "What I meant is shouldn't there be a question number C1.1, C1.1b c1.1c as in the pdf? When I look at data I see only C1.1a it seems that those other questions that are present in the pdf \"2020_Climate_Change_Questionnaire\" but the data only contains answers to question C1.1a. Try run the following and compare with the pdf questionnaire structure for the governance section:\n\nimport pandas as pd\nq2020_crop = pd.read_csv('../input/cdp-unlocking-climate-solutions/Corporations/Corporations Responses/Climate Change/2020_Full_Climate_Change_Dataset.csv')\npd.options.display.max_rows = 1000\nq2020_crop.groupby([\"question_number\"]).size()", "1062517": "Or better, use this code:\n\nimport pandas as pd\nq2020_crop = pd.read_csv('../input/cdp-unlocking-climate-solutions/Corporations/Corporations Responses/Climate Change/2020_Full_Climate_Change_Dataset.csv')\npd.options.display.max_rows = 1000\nq2020_crop.groupby([\"module_name\",\"question_number\"]).size()", "1063159": "The file on **cities** is correct with respect to the pdf questionnaire.\nBut **for corporations**, some modules are missing when comparing the pdf and csv files", "1063309": "Thanks for the questions!\n\nDue to compliance and legal restrictions, we are only able to include a limited amount of question responses within the corporate data sets available to you in the .csvs. \n\nSo although you can see ALL the questions that company's are presented with in the PDFs, you are only provided with question responses that we are legally permitted to share publicly within the .csvs.\n\n*e.g. that's why you can only find responses to C1.1a within the dataset - we are not permitted to publicly release the responses to to C1.1, C1.1b or C1.1c within the context of this competitions dataset*\n\nThe PDFs are there to give you more contextual knowledge around how our questions are designed, but include questions that you do not have response data for.\n\nHope this clears things up!", "1063343": "As mentioned by Callum [above](https://www.kaggle.com/c/cdp-unlocking-climate-solutions/discussion/193612#1063309) (just recently), a subset of data points from the corporate disclosures have been redacted to ensure legal compliance.", "1063384": "Many thanks for your prompt reply. This is really helpful." }, "source": "meta" }