{"cells":[{"metadata":{},"cell_type":"markdown","source":"<img src=\"https://storage.googleapis.com/kaggle-competitions/kaggle/22420/logos/header.png\">\n<br>\nDear Readers, <br><br>\nThank you for taking the time to read this notebook. Due to the systemic nature of this topic, each theme and metric has been discussed in a comprehensive manner in order to fully visualise the magnitude and effects of climate change on inequality. Enjoy and welcome to the portal!  \n<br><br>\n\n<h1 style=\"color: #FFFFFF\">Table of Contents</h1>\n\n* [1. Executive Summary](#executive-summary)\n* [2. Introduction](#introduction)\n    * [2.1 Carbon Disclosure Project](#cdp)\n    * [2.2 21st Century Problem](#21st-century-problem)\n    * [2.3 Journey through The Portal](#journey-through-the-portal)\n* [3. Problem Statement](#problem-statement)\n* [4. Methodology](#methodology)\n* [5. Datasets Used](#datasets-used)\n* [6. Vulnerability Assessment](#vulnerability-assessment)\n    * [6.1 City Vulnerability](#city-vulnerability)\n        * [6.1.1 Vulnerability and Preparedness](#vulnerability-and-preparedness)\n        * [6.1.2 Adaptation](#adaptation)\n    * [6.2 Corporate Vulnerability](#corporate-vulnerability)\n* [7. Climate Impact](#climate-impact)\n    - [7.1 Emissions](#emissions)\n        * [7.1.1 City Emissions](#city-emissions)\n        * [7.1.2 Corporate Emissions](#corporate-emissions)\n    - [7.2 Energy](#energy)\n        * [7.2.1 Renewable Energy](#renewable-energy)\n        * [7.2.2 Corporate Energy Consumption](#corporate-energy-consumption)\n    - [7.3 Water Risks](#water-risks)\n        * [7.3.1 Hidden Water Costs of Electricity](#water-costs-of-electricity)\n        * [7.3.2 Water Stress](#water-stress)\n* [8. Collaboration](#collaboration)\n* [9. Key Takeaways](#key-takeaways)"},{"metadata":{"_kg_hide-output":true,"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"%%HTML\n<style type=\"text/css\">\n\nh1 {\n    text-align: center; \n    background-color: #D10E40; \n    padding: 5px; \n    font-size: 30px; \n}\n</style>","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":false},"cell_type":"code","source":"import re\n\nimport numpy as np  # linear algebra\nimport pandas as pd  # data processing, CSV file I/O (e.g. pd.read_csv)\npd.set_option('display.max_rows', 500)\npd.set_option('display.max_columns', None)\npd.options.mode.chained_assignment = None\n\nimport gensim\nfrom tqdm import tqdm\nimport ipywidgets as widgets\nimport seaborn as sns\nimport matplotlib.pyplot as plt\nimport plotly_express as px\n\nsns.set()\nplt.rcParams.update({\n    'figure.figsize': (16, 13),\n    'axes.titlesize' : 20,\n    'axes.titlepad': 18,\n    'axes.labelsize': 18,\n    'axes.labelpad': 20,\n    'xtick.labelsize': 15,\n    'ytick.labelsize': 15,\n\n#     'figure.titleweight': 'heavy',\n#     'figure.edgecolor': 'k',\n#     'lines.linewidth' : 3,\n#     'image.cmap': 'Reds'\n})\n\nfrom wordcloud import WordCloud\nfrom nltk.corpus import stopwords\nfrom sklearn.feature_extraction.text import CountVectorizer\nfrom sklearn.decomposition import LatentDirichletAllocation as LDA\n\nimport warnings\nwarnings.filterwarnings(\"ignore\")\n\nstop = stopwords.words('english') + stopwords.words('spanish') + stopwords.words('french')\n\ngrey = '#828689'\ngrad3 = ['#E1A29F', '#D10E40', '#9A131F']\ngrad4 = ['#62194A', '#9A131F', '#D10E40', '#E1A29F']\nqualitative = ['#D10E40', '#3E779D', '#6C164D', '#9A131F', '#C54227', '#90619B', '#DA4F57', '#477F67', '#EB7023', '#E1A29F']\n\npalette = ['#D10E40', '#4E8AC9', '#EB7023']\npalette_r = list(reversed(palette))\n\n# grey = '#979EA4'\n# palette = ['#C41102', '#4E8AC9', '#E76335']\n# palette_r = ['#E76335', '#4E8AC9', '#C41102']\n# palette2 = [\"#7D0744\", \"#4E8AC9\", '#E76335']\nCITY_GROUP = ['CDP Region', 'Country', 'City', 'Account Number']","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<h1  style=\"color: #FFFFFF\" id=\"executive-summary\">1 Executive Summary</h1>\n<br>\n\nThe following report is a submission to the CDP: Unlocking Climate Solutions Kaggle Competition. Over the course of this competition, we have learnt a lot, not only from the intensive research required to tackle the problem statement, but also from analysing the vast repository of data CDP made available.\n\nThe objective of this report was to analyse the issues of inequality while developing KPIs that address climate change. Our key findings related to these issues are as follows:\n- Most cities face medium to high levels of climate exposure, resulting in effects that will be felt universally, however, the poorest and elderly populations will see a disproportionate amount of the impact directed towards them.\n- These impacts will lead to significant increases in crime and conflict, and fluctuating socioeconomic conditions. These will be closely related to increases in demand for resources.\n- Climate change will also play a significant role in aggravating health issues, particularly for those already vulnerable, which will put additional stress on and disrupt healthcare services due to increasing demands.\n- Cities facing high levels of exposure can adopt relevant adaptation actions to build resilience to battle climate risks. However, a number of cities face serious shortages of resources  which may seriously hamper their ability to adapt.\n- Cities can be incentivized to adopt early adaptation efforts by identifying parallel opportunities for social and economic benefit.\n- A number of cities face serious shortages of resources which may seriously hamper their ability to adapt.\n- There is a bilateral relationship between the impact corporations have on the environment, and the level of financial liability they face from climate change. Further, there is a correlation between the climate risks they face and the broader social issues that co-occur with these risks.\n- There is a noticeable lack of questions addressing the social issues related to, and arising from, climate change in the CDP Corporations Climate Change Questionnaire.\n- There are strong positive correlations between emissions and GDP, but also more developed cities find it easier to implement emission reduction actions - highlighting how they should take a leading role in tackling Global Warming.\n- Corporations in the Fossil Fuel/Energy and Transportation industries contribute inordinately towards GHG emissions, due to which they have a greater potential of reducing emissions.\n- There is currently a divide between the developed and developing world in the amount of energy generated from renewable sources. Due to unequal conditions, poorer cities may find it difficult to install sufficient sources of renewable energy to balance out their use of unclean fuels, and thereby have to depend even more on fossil fuels.\n- A small proportion of all corporations, such as those in the manufacturing and materials industries consume much higher proportions of electricity than others, shedding light on the disparity in available resources in electricity poor parts of the world.\n- Corporate fuel use is a direct cause of adverse effects on human health, and can be connected to deaths due to air pollution and accidents in low visibility conditions. \n- Electricity generation for corporate use results in significant amounts of water consumption. Corporations, especially those part of the energy sector, must pay special attention to this to avoid worsening water related issues.\n- A significant number of corporate facilities source their water from resources that are water-stressed or located in water-stressed regions, worsening the problems related to water-scarcity.\n- The existence of shared opportunities and risks between cities and corporations presents added incentives for the two to work together in solving key climate issues.\n\n\n<h2 style=\"text-align:center\">KPI Summary Table</h2>"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"metric_table = [\n    [\"City Vulnerability\", \n     \"The susceptibility of cities to the possible risks and negative impacts of climate change, due to inadequate preparedness, high exposure, or both.\", \n     \"$Vulnerability_{city} = \\\\sum_{i=1}^{climateHazards}max(0, ClimateExposure_{i} - Preparedness_{i})$\", \"\", \"\", \"\"],\n    [\"Adaptation Incentive\", \n     \"Motive for cities to adapt to climate change.\", \n     \"$Adaptation Incentive_{city} = \\\\sum_{i=1}^{climate Hazards}(Vulnerability_{i} + OpportunityPayoff_{i})$\", \"\", \"\", \"\"],\n    [\"Adaptability\", \n     \"Ability of a city to adapt to climate change.\", \n     \"$Adaptability_{city} = AdaptabilityFactor_{city} \\\\times Proactiveness_{city} \\\\times (10 \\\\times HumanCapitalScore + \\\\log_{10}(1 + Funding))$\", \"\", \"\", \"\"],\n    [\"Climate Liability\", \n     \"A measure of the total risk to corporations from aggregated financial impacts of climate change.\", \n     \"$ClimateLiability_{org} = \\\\sum_{i=1}^{risks}(LiabilityFactor_{i} \\\\times TotalCost_{i})$\", \"\", \"\", \"\"],\n    [\"Lifetime Mitigation Score\", \n     \"Lifetime performance (or utility) of a city's emission mitigation action at reducing direct and indirect emissions of CO2 equivalencies.\", \n     \"$LifetimeMitigationScore = \\\\log_{10}(1 + DirectEmissionReduction_{metricTonnesCO2e}) + \\\\log_{10}(1 + IndirectEmissionReduction_{metricTonnesCO2e}) - \\\\log_{10}(1 + CostOfProject)$\", \"\", \"\", \"\"],\n    [\"Reduction Initiative Score\", \n     \"The utility of a corporate emission reduction initiative at reducing CO2 equivalent emissions.\", \n     \"$ReductionInitiativeScore_{lifetime} = \\\\log_{10}(1 + AnnualCO_{2}eSavings \\\\times Lifetime) + \\\\log_{10}(1 + AnnualMonetarySavings \\\\times Lifetime) - \\\\log_{10}(1 + TotalLifetimeInvestment)$\", \"\", \"\", \"\"],\n    [\"Renewable Energy Public Demand\", \n     \"Years of renewable energy generation required, at the current capacity, to meet annual energy demand for a city.\", \n     \"$Demand_{years} = \\\\frac{TotalEnergyUse_{MWh}}{RenewableEnergyCapacity_{MW} \\\\times 8760}$\", \"\", \"\", \"\"],\n    [\"Corporate Electricity Greed\", \n     \"Measure of corporate electricity consumption as a percentage of the global electricity poverty.\", \n     \"$CorporateElectricityGreed = \\\\frac{Total Electricity Consumption_{MWh}}{GlobalElectricityPoverty} \\\\times 100$\", \"\", \"\", \"\"],\n    [\"Deaths by Corporate Fuel Use\", \n     \"Number of deaths caused due to corporations' choice of fuels for energy generation.\", \n     \"$NoOfDeaths = \\\\sum_{i=1}^{fuels}Fuel Consumption(TWh)_{i} \\\\times DeathsPerTWh_{i}$\", \"\", \"\", \"\"],\n    [\"Water Cost of Electricity\", \n     \"Gallons of upstream water consumed for purposes relating to the generation of electricity.\", \n     \"$Water Cost_{gallons} = \\\\sum_{i=1}^{fuels}ElectricityConsumed(kWh)_{i} \\\\times WaterConsumptionFactor_{i}$\", \"\", \"\", \"\"],\n    [\"Unsustainable Water Use\", \n     \"Measure of water consumption from water-stressed regions and resources.\", \n     \"$UnsustainableWaterUse = \\\\sum_{i=1}^{facilites}(WaterStressScore_{i} \\\\times \\\\log_{10}(1 + WaterConsumption_{i}))$\", \"\", \"\", \"\"],\n    [\"Collaboration Strength\", \n     \"A measure of the joint interest and expected mutual benefits from collaborative partnerships between cities and corporations on shared risks, opportunities, and initiatives.\", \n     \"$CollaborationStrength_{city,org}  = SharedRisk \\\\times CommonOpportunities \\\\times \\\\mathtt{SERO}Score_{city,org}$\", \"\", \"\", \"\"]\n]\n\nchildren= []\nfor i, row in enumerate(metric_table):\n    display = ''\n    for j, col in enumerate(row[1:3]):\n        if j==0:\n            display += f\"<i>{col}</i><br><br>\"\n        elif j==1:\n            display += col + \"<br><br>\"\n    display += f'<a href=\"https://www.kaggle.com/mannmann2/kpis-for-measuring-climate-action-and-inequality#metric{i+1}\">Go to metric</a>'\n    children.append(widgets.HTMLMath(value=display))\n                        \naccordian = widgets.Accordion(children=children)\n\nfor i in range(len(children)):\n    accordian.set_title(i, f\"Metric {i+1}: {metric_table[i][0]}\")\n\naccordian","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<h1 style=\"color: #FFFFFF\" id=\"introduction\">2 Introduction</h1>  \n\n<h2 id=\"cdp\">2.1 Carbon Disclosure Project</h2>\n\nAs the world tries to tackle a global pandemic, cities and corporations, large and small are focused on reassessing their long term and short term growth. There exists a strong focus on the societal impact that climate change plays in order for communities to thrive and survive. The long term lens has made it imperative now more than ever to look beyond and create a sustainable future, a future where cities and businesses work together to evaluate, rehabilitate and safeguard the world. \n\n[Carbon Disclosure Project (CDP)](https://www.cdp.net/en/info/about-us) a not-for-profit charity is working on accelerating an economic transition that works for the people and planet in the long term. CDP runs the global disclosure system for investors, companies, cities, states and regions to manage their environmental impacts on climate change, water security and deforestation. The disclosure system tracks the thousands of companies, cities, states and regions that supply information in its annual reporting process, and scores companies and cities based on their journey through disclosure and towards environmental leadership. The world’s economy looks to CDP as the gold standard of environmental reporting with the richest and most comprehensive dataset on corporate and city action to measure and manage their risks and opportunities.\n\n<h2 id=\"21st-century-problem\">2.2 21st Century Problem</h2>\n\nThe data provided through the Questionnaires by CDP gives insight into the environmental impact of cities and corporations. As the data continues to grow, there is great potential in how these insights could be used in a consequential way. According to the [United Nations Human Development Report, 2019](http://report.hdr.undp.org/), there exists an important issue which is often ignored when we consider the emerging and existing risks that we will face in the 21st century - the interwovenness of Climate Change and Inequality. Through this report, we aim to focus on the importance of integrating social equity into climate action. Using data from the questionnaires along with external datasets we aim to develop actionable Key Performance Indicators that can be adopted to track climate action geared towards solving social and environmental issues.\n \n<h2 id=\"journey-through-the-portal\">2.3 Journey through The Portal</h2>\n\nWe start this report by presenting the problem statement as defined by CDP/Kaggle (Section 3).\n\nWe then move on to an explanation of our framework and methodology for developing KPIs (Section 4). Once these have been introduced, we provide a short overview of the various datasets used throughout the notebook (Section 5), and then dive into building our KPIs (Sections 6, 7, 8). The focus of our KPIs is on 3 main areas. \n\n- In Section 6, we look at Vulnerability and establish ways to measure the impact of climate change on cities and corporations, while discussing the broader social implications of these risks.\n- In Section 7, we look at 3 individual themes - Emissions, Energy, and Water - relating to Climate Impact, and assess the effect activities undertaken in these areas may have on making a more equitable society, or inversely, furthering social inequality and division.\n- In Section 8, we suggest a way to measure the Collaboration ability of cities and organizations to partner with one another and work together on mitigating shared risks and realizing their common goals and ambitions.\n\nFinally, we offer our final recommendations and closing thoughts in the Key Takeaways (Section 9).\n\n<h1 style=\"color: #FFFFFF\" id=\"problem-statement\">3 Problem Statement</h1>\n\n<br>\n\nAs per the [competition website](https://www.kaggle.com/c/cdp-unlocking-climate-solutions), the problem statement is as follows:\n\nDevelop a methodology for calculating key performance indicators (KPIs) that relate to the environmental and social issues that are discussed in the CDP survey data. Leverage external data sources and thoroughly discuss the intersection between environmental issues and social issues. Mine information to create automated insight generation demonstrating whether the city and corporate ambitions take these factors into account.\n\nThe website also provides a set of questions that could be used as guidelines when attempting a solution to the problem statement.\n\n- How do you help cities adapt to a rapidly changing climate amidst a global pandemic, but do it in a socially equitable way?\n- What are the projects that can be invested in that will help pull cities out of a recession, mitigate climate issues, but not perpetuate racial/social inequities?\n- What are the practical and actionable points where city and corporate ambition join, i.e. where do cities have problems that corporations affected by those problems could solve, and vice versa?\n- How can we measure the intersection between environmental risks and social equity, as a contributor to resiliency?\n\n<h1 style=\"color: #FFFFFF\" id=\"methodology\">4 Methodology</h1>\n\n<img src=\"https://i.pinimg.com/564x/1b/cb/87/1bcb8744497dca3ac1f44a5540cb3a58.jpg\" width=600><br>\n\nClimate Change and Social inequality are extremely complex issues that require a holistic approach to developing solutions. The current movement to transition to net-zero, at its core is a collective good, but there will always be disagreements and debates when it comes to creating a direction for sustainability transitions due to the ambiguous and contested nature of the concept. Keeping this in mind, we realised that to push for ambitious actions towards a collective good, we had to develop metrics that are able to prove accountability across our three main themes of Vulnerability, Climate Impact and Collaboration. Thereafter, using the data from the CDP questionnaires and a few of the external datasets, we were able to establish a relationship between inequality (in various contexts) and climate change, and bring to light unique properties which may not be visible when looking at the constituent parts separately from a reductionist approach.\n\nThe systemic nature of climate change based inequality required us to zoom out and look at its functioning as a whole unit. This is why we decided to apply a broad lens and analyse the data at the global level - because we understand that the performance of climate action and the nature of inequality varies in every country based on human capital, resources, technological advancements etc, and focusing with a narrow lens would give room for implicit biases to play out. This approach resulted in many of our personal biases not being justified when it came to looking at the data, in fact we saw them crash before our eyes, and this allowed us to gain new insights which contributed to helping us build _impact_ metrics that could be easily compared between cities and corporations for different regions.\n\nWhile the KPIs we developed were not exclusive to the S.M.A.R.T (Specific, Measurable, Attainable, Relevant, Time-bound) approach, this principle was crucial in building a foundation when it came to the process of selecting relevant metrics.\n\n<h1 style=\"color: #FFFFFF\" id=\"datasets-used\">5 Datasets Used</h1>"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# using the given city administative organizations in the disclosure data we create a mapping between \n# account numbers and city names for some of the cities which do not have a value in the 'City' column\n# this will be used to match OECD economic data with the CDP cities \nACCOUNT_MAPPING = {\n    74643: 'Dijon',\n    50377: 'Queretaro',\n    74607: 'Alkmaar',\n    833284: 'West Midlands urban area',\n    50540: 'Albuquerque',\n    74575: 'Dane',\n    74534: 'Erie (NY)',\n    74563: 'Guilford',\n    74546: 'Milwaukee',\n    832610: 'Orange',\n    59531: 'Santa Barbara',\n    74558: 'Summit',\n    834373: 'York'\n}\n\n# used in city_gdp\nCOUNTRY_CODES = {\n    'AT': 'Austria',\n    'AUS': 'Australia',\n    'BE': 'Belgium',\n    'CAN': 'Canada',\n    'CH': 'Switzerland',\n    'CL': 'Chile',\n    'CZ': 'Czech Republic',\n    'DE': 'Germany',\n    'DK': 'Denmark',\n    'EE': 'Estonia',\n    'EL': 'Greece',\n    'ES': 'Spain',\n    'FI': 'Finland',\n    'FR': 'France',\n    'HU': 'Hungary',\n    'IE': 'Ireland',\n    'IT': 'Italy',\n    'JPN': 'Japan',\n    'KOR': 'Republic of Korea',\n    'LT': 'Lithuania',\n    'LU': 'Luxembourg',\n    'LV': 'Latvia',\n    'MEX': 'Mexico',\n    'NL': 'Netherlands',\n    'NO': 'Norway',\n    'PL': 'Poland',\n    'PT': 'Portugal',\n    'SE': 'Sweden',\n    'SI': 'Slovenia',\n    'SK': 'Slovakia',\n    'UK': 'United Kingdom of Great Britain and Northern Ireland',\n    'USA': 'United States of America'\n}\n\n# used in city_gdp\nNORM_CITY_NAMES = {\n    'Montreal': 'Montréal',\n    'Zurich': 'Zürich',\n    'Mannheim-Ludwigshafen': 'Mannheim',\n    'Malaga': 'Málaga',\n    'Vitoria': 'Vitoria-Gasteiz',\n    'Florence': 'Firenze',\n    'Genoa': 'Genova',\n    'Milan': 'Milano',\n    'Naples': 'Napoli',\n    'Padua': 'Padova',\n    'Reggio nell\\'Emilia': 'Reggio Emilia',\n    'Rome': 'Roma',\n    'Turin': 'Torino',\n    'Venice': 'Venezia',\n    'Merida': 'Mérida',\n    'Gdansk': 'Gdańsk',\n    'Gothenburg': 'Göteborg',\n    'Greater Adelaide': 'Adelaide',\n    'Greater Melbourne': 'City of Melbourne',\n    'Greater Perth': 'Perth',\n    'Greater Sydney': 'City of Sydney',\n    'Malmo': 'Malmö',\n    'West Midlands urban area': 'Birmingham',\n    'Detroit (Greater)': 'Detroit',\n    'Durham': 'Durham, NC',\n    'Fresno (Greater)': 'Fresno',\n    'Lancaster (PA)': 'Lancaster, PA',\n    'Los Angeles (Greater)': 'Los Angeles',\n    'Miami (Greater)': 'Miami',\n    'New York (Greater)': 'New York City',\n    'Philadelphia (Greater)': 'Philadelphia',\n    'Portland': 'Portland, OR',\n    'Richmond (Greater)': 'Richmond, VA',\n    'Rochester (NY)': 'Rochester',\n    'Salt Lake': 'Salt Lake City',\n    'San Francisco (Greater)': 'San Francisco',\n    'Santa Cruz': 'Santa Cruz, CA',\n    'St. Louis': 'St Louis',\n    'Washington (Greater)': 'Washington, DC'\n}\n\n# used in energy_use\nCOUNTRY_NAMES = {\n    'Bolivia': 'Bolivia (Plurinational State of)',\n    'Democratic Republic of Congo': 'Democratic Republic of the Congo',\n    'Hong Kong': 'China, Hong Kong Special Administrative Region',\n    'Moldova': 'Republic of Moldova',\n    'Palestine': 'State of Palestine',\n    'Russia': 'Russian Federation',\n    'South Korea': 'Republic of Korea',\n    'Taiwan': 'Taiwan, Greater China',\n    'United Kingdom': 'United Kingdom of Great Britain and Northern Ireland',\n    'United States': 'United States of America',\n    'Venezuela': 'Venezuela (Bolivarian Republic of)',\n    'Vietnam': 'Viet Nam'\n}\n\n# used in corp_locations\nSTATE_CODES = {\n    'Alabama': 'AL',\n    'Alaska': 'AK',\n    'American Samoa': 'AS',\n    'Arizona': 'AZ',\n    'Arkansas': 'AR',\n    'Ca': 'CA',\n    'California': 'CA',\n    'Colorado': 'CO',\n    'COLORADO': 'CO',\n    'Connecticut': 'CT',\n    'Delaware': 'DE',\n    'District of Columbia': 'DC',\n    'Florida': 'FL',\n    'Georgia': 'GA',\n    'Guam': 'GU',\n    'Hawaii': 'HI',\n    'Idaho': 'ID',\n    'Illinois': 'IL',\n    'Indiana': 'IN',\n    'Iowa': 'IA',\n    'Kansas': 'KS',\n    'Kentucky': 'KY',\n    'Louisiana': 'LA',\n    'Maine': 'ME',\n    'Maryland': 'MD',\n    'Massachusetts': 'MA',\n    'Michigan': 'MI',\n    'Minnesota': 'MN',\n    'Mississippi': 'MS',\n    'Missouri': 'MO',\n    'Montana': 'MT',\n    'Nebraska': 'NE',\n    'Nevada': 'NV',\n    'New Hampshire': 'NH',\n    'New Jersey': 'NJ',\n    'New Mexico': 'NM',\n    'New York': 'NY',\n    'North Carolina': 'NC',\n    'North Dakota': 'ND',\n    'Northern Mariana Islands':'MP',\n    'Ohio': 'OH',\n    'Oklahoma': 'OK',\n    'Oregon': 'OR',\n    'Pennsylvania': 'PA',\n    'Puerto Rico': 'PR',\n    'Rhode Island': 'RI',\n    'South Carolina': 'SC',\n    'South Dakota': 'SD',\n    'Tennessee': 'TN',\n    'Texas': 'TX',\n    'Utah': 'UT',\n    'Vermont': 'VT',\n    'Virgin Islands': 'VI',\n    'Virginia': 'VA',\n    'Washington': 'WA',\n    'West Virginia': 'WV',\n    'Wisconsin': 'WI',\n    'Wyoming': 'WY',\n    'ALBERTA': 'AB',\n    'Quebec': 'QC',\n    'Manitoba': 'MB',\n    'Ontario': 'ON'\n}","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"1. The **CDP: Unlocking Climate Solutions** [Competition Data](https://www.kaggle.com/c/cdp-unlocking-climate-solutions/data): The following sub-classes of the data provided by CDP have been used throughout this notebook.\n    - Cities Disclosing\n    - Cities Responses\n    - Corporations Disclosing\n    - Corporations Responses\n    - Locations of Corporations\n1. [Human Capital Index (HCI) 2020](https://datacatalog.worldbank.org/dataset/human-capital-index) (World Bank): This dataset provides a measure of the human capital expected to be available \"given the risks of poor health and poor education prevailing in the country.\"\n1. [Corporate Environmental Impact](https://www.hbs.edu/impact-weighted-accounts/Pages/default.aspx) (HBS - Impact Weighted Accounts): Provides monetization factors for measuring the total environmental impact of corporations in dollars. \n1. [Economic Indicators for OECD Cities](https://stats.oecd.org/Index.aspx?DataSetCode=CITIES): Provides key economic indicators like GDP and GDP per capita for OECD cities.\n1. Energy data from [Our World in Data](https://ourworldindata.org/): Provides a number of statistics a relating to energy access and consumption.\n    - [Per capita Energy Consumption](https://ourworldindata.org/grapher/per-capita-energy-use)\n    - [People Without Electricity](https://ourworldindata.org/energy-access)\n    - [Per capita Electricity Consumption](https://ourworldindata.org/grapher/per-capita-electricity-consumption)\n1. Water related data from [WRI](https://www.wri.org):\n    - [Water Use Embedded in Purchased Electricity](https://www.wri.org/resources/data-sets/dataset-guidance-calculating-water-use-embedded-purchased-electricity): Provide a way to calculate water consumption during eleectricity generation from common fuels.\n    - [Aqueduct Baseline Water Stress](https://www.wri.org/publication/aqueduct-country-and-river-basin-rankings): Ranks countries and major river basins by their level of water stress.\n1. Additional Public Datasets from Kaggle:\n    - [Annual Financial Data for Hybrid Metrics](https://www.kaggle.com/takahirokubo0/annual-financial-data-for-hybrid-cdp-kpi) (SimFin): Provides financial metics, such as Revenue and EBITDA, for corporates disclosing to CDP.\n    - [ISO-3166 Country Mappings](https://www.kaggle.com/andradaolteanu/country-mapping-iso-continent-region): Provides a 3 digit iso codes for counties and regions to facilite choropleth geomaps in plotly.\n    - [Google's Word2Vec Vectors](https://www.kaggle.com/umbertogriffo/googles-trained-word2vec-model-in-python): Pre-trained word embeddings from Google, trained on a 100 billion word dataset from Google News."},{"metadata":{"_kg_hide-input":true,"trusted":true,"_kg_hide-output":true},"cell_type":"code","source":"# LOAD CITY DATA\nPATH = '../input/cdp-unlocking-climate-solutions/Cities/'\n\n# disclosing data\ncd2020 = pd.read_csv(PATH + \"Cities Disclosing/2020_Cities_Disclosing_to_CDP.csv\")\ncd2019 = pd.read_csv(PATH + \"Cities Disclosing/2019_Cities_Disclosing_to_CDP.csv\")\ncd2018 = pd.read_csv(PATH + \"Cities Disclosing/2018_Cities_Disclosing_to_CDP.csv\")\ncities_dis = pd.concat([cd2020, cd2019, cd2018])\n\n# getting city name from ACCOUNT MAPPING\ncities_dis['City'] = cities_dis['City'].fillna(cities_dis['Account Number'].map(ACCOUNT_MAPPING))\n\n# response data\ncr2020 = pd.read_csv(PATH + \"Cities Responses/2020_Full_Cities_Dataset.csv\")\ncr2019 = pd.read_csv(PATH + \"Cities Responses/2019_Full_Cities_Dataset.csv\")\ncr2018 = pd.read_csv(PATH + \"Cities Responses/2018_Full_Cities_Dataset.csv\")\ncities_res = pd.concat([cr2020, cr2019, cr2018])\n\n# merging disclosing data with response data\ncities = pd.merge(cities_dis, cities_res, 'inner', on=['Year Reported to CDP', 'Account Number', 'Organization', 'Country', 'CDP Region',])\n# the following columns contain data we do not need, and are dropped\ncities.drop(columns=['Access', 'File Name', 'Last update_x', 'Last update_y'], inplace=True)\n\n\n# creating population dataframe using the most recent values present in the disclosure data\npopulation = cities_dis[['City', 'Account Number', 'Year Reported to CDP', 'Population']].dropna(subset=['Population']).reset_index(drop=True)\npopulation = population.loc[population.groupby(['City', 'Account Number'])['Year Reported to CDP'].idxmax()].set_index(['City', 'Account Number'])['Population']\n\n\n# creating a city location dataframe to facilitate geo-plotting \ncity_locations = cities_dis[['Country', 'City', 'Account Number', 'City Location']].drop_duplicates().dropna(subset=['City Location']).set_index('City')\ncity_locations[['lon', 'lat']] = city_locations['City Location'].apply(lambda x: pd.Series(x.split(\"POINT \")[1].strip('()').split())).astype('float32')\n\n#fixing inverted coordinates\nfor org in ['Mississauga', 'Norfolk', 'Aurora', 'South Bend', 'San José']:\n    city_locations.at[org, 'lat'], city_locations.at[org, 'lon'] = city_locations.loc[org]['lon'], city_locations.loc[org]['lat']\n# Aurora also has the negative sgin missing\ncity_locations.at['Aurora', 'lon'] = -city_locations.loc['Aurora']['lon']\ncity_locations = city_locations.set_index('Country', append=True)\n\n\n# deleting temperory variables\ndel cities_res, cd2018, cd2019, cr2018, cr2019, cr2020\n# ------------------------------------------------------\n\n# LOAD CLIMATE DATA\nPATH2 = '../input/cdp-unlocking-climate-solutions/Corporations/'\n\n# disclosing\ncd_cc2018 = pd.read_csv(PATH2 + 'Corporations Disclosing/Climate Change/2018_Corporates_Disclosing_to_CDP_Climate_Change.csv')\ncd_cc2019 = pd.read_csv(PATH2 + 'Corporations Disclosing/Climate Change/2019_Corporates_Disclosing_to_CDP_Climate_Change.csv')\ncd_cc2020 = pd.read_csv(PATH2 + 'Corporations Disclosing/Climate Change/2020_Corporates_Disclosing_to_CDP_Climate_Change.csv')\ncc_dis = pd.concat([cd_cc2020, cd_cc2019, cd_cc2018])\n\n# responses\ncr_cc2018 = pd.read_csv(PATH2 + 'Corporations Responses/Climate Change/2018_Full_Climate_Change_Dataset.csv')\ncr_cc2019 = pd.read_csv(PATH2 + 'Corporations Responses/Climate Change/2019_Full_Climate_Change_Dataset.csv')\ncr_cc2020 = pd.read_csv(PATH2 + 'Corporations Responses/Climate Change/2020_Full_Climate_Change_Dataset.csv')\ncc_res = pd.concat([cr_cc2020, cr_cc2019, cr_cc2018])\n\n# merge disclosing data with responses\nclimate = pd.merge(cc_dis, cc_res, 'inner', on=['survey_year', 'account_number', 'organization', 'response_received_date'])\n\n# delete temporary variables\ndel cc_res, cd_cc2018, cd_cc2019, cd_cc2020, cr_cc2018, cr_cc2019, cr_cc2020\n# ------------------------------------------------------\n\n# LOAD WATER SECURITY DATA\nPATH2 = '../input/cdp-unlocking-climate-solutions/Corporations/'\n\n# disclosing\ncd_ws2018 = pd.read_csv(PATH2 + 'Corporations Disclosing/Water Security/2018_Corporates_Disclosing_to_CDP_Water_Security.csv')\ncd_ws2019 = pd.read_csv(PATH2 + 'Corporations Disclosing/Water Security/2019_Corporates_Disclosing_to_CDP_Water_Security.csv')\ncd_ws2020 = pd.read_csv(PATH2 + 'Corporations Disclosing/Water Security/2020_Corporates_Disclosing_to_CDP_Water_Security.csv')\nws_dis = pd.concat([cd_ws2020, cd_ws2019, cd_ws2018])\n\n# responses\ncr_ws2018 = pd.read_csv(PATH2 + 'Corporations Responses/Water Security/2018_Full_Water_Security_Dataset.csv')\ncr_ws2019 = pd.read_csv(PATH2 + 'Corporations Responses/Water Security/2019_Full_Water_Security_Dataset.csv')\ncr_ws2020 = pd.read_csv(PATH2 + 'Corporations Responses/Water Security/2020_Full_Water_Security_Dataset.csv')\nws_res = pd.concat([cr_ws2020, cr_ws2019, cr_ws2018])\n\n# merge disclosing and response data\nwater = pd.merge(ws_dis, ws_res, 'inner', on=['survey_year', 'account_number', 'organization', 'response_received_date'])\n\n# delete temporary variables\ndel ws_res, cd_ws2018, cd_ws2019, cd_ws2020, cr_ws2018, cr_ws2019, cr_ws2020\n# ------------------------------------------------------\n\n# loading national human capital index data\nhuman_capital = pd.read_csv('../input/human-capital-index-2020-the-world-bank/hci_MaleFemale_september_2020.csv').replace('-', np.nan)\n\n# normalize country names as per CDP values\nhuman_capital['Country Name'] = human_capital.replace({\n    'Bolivia': 'Bolivia (Plurinational State of)',\n    'Hong Kong SAR, China': 'China, Hong Kong Special Administrative Region',\n    'Gb': 'Gibraltar',\n    'Mn': 'Monaco',\n    'Korea, Rep.': 'Republic of Korea',\n    'Palestine': 'State of Palestine',\n    'Taiwan': 'Taiwan, Greater China',\n    'United Kingdom': 'United Kingdom of Great Britain and Northern Ireland',\n    'Tanzania': 'United Republic of Tanzania',\n    'United States': 'United States of America',\n})\n\nhuman_capital.set_index(['Country Name'], inplace=True)\n# ------------------------------------------------------\n\n# loading corporate environmental impact data\ncorp_env_impact = pd.read_csv('../input/corporate-environmental-impact/final_raw_sample_0_percent.csv')\n# filter on US and Canadian companies\ncorp_env_impact = corp_env_impact[corp_env_impact['Country'].isin(['United States', 'Canada'])]\n# select most recent year's data\ncorp_env_impact = corp_env_impact.loc[corp_env_impact.groupby('Company Name')['Year'].idxmax()]\n\n# cleaning corporation names to facilitate matching with CDP data\ncorp_env_impact['cleaned_name'] = corp_env_impact['Company Name'].str.lower() \\\n.replace(re.compile('[, ](co|corp|cos|llc|ltd|inc)\\.'), '') \\\n.replace(re.compile('[, ](co|corp|company|corporation|cos|plc|llc|ltd|incorporation|incorporated|inc)$'), '') \\\n.replace(re.compile(\"[!\\.,\\\"'&]\"), '').str.replace('-', ' ') \\\n.replace(re.compile('^the '), '').replace(re.compile('\\s+'), ' ').str.strip()\n\ncorp_env_impact['Country'] = corp_env_impact['Country'].replace({'United States': 'United States of America'})\ncorp_env_impact.set_index(['Country', 'cleaned_name'], inplace=True)\n# ------------------------------------------------------\n\n# oecd metropolitan gdp dataset\ncity_gdp = pd.read_csv('../input/economic-indicators-for-oecd-metropolitan-areas/CITIES_17112020134646826.csv')\n# normalize city names and get country from codes\ncity_gdp['City'] = city_gdp['Metropolitan areas'].replace(NORM_CITY_NAMES)\ncity_gdp['Country'] = city_gdp['METRO_ID'].str.rstrip('0123456789').map(COUNTRY_CODES)\n\ncity_gdp = city_gdp.drop('Metropolitan areas', axis=1) \\\n                   .pivot(index=['Country', 'City', 'Year'], columns='VAR', values=['Value'])['Value'].reset_index()\n\n# selecting most recent data by extracting for max value for year for each city\ncity_gdp = city_gdp.loc[city_gdp.groupby(['Country', 'City'])['Year'].idxmax()].set_index(['Country', 'City'])\n# ------------------------------------------------------\n\n# loading per capita energy use data\nenergy_use = pd.read_csv('../input/world-per-capita-energy-consumption/per-capita-energy-use.csv')\n# rename countries as per CDP\nenergy_use['Country'] = energy_use['Entity'].replace(COUNTRY_NAMES)\n# select latest data for each country\nenergy_use = energy_use.loc[energy_use.groupby('Code')['Year'].idxmax()].drop(['Entity', 'Year'], axis=1).set_index('Country').sort_index()\n# ------------------------------------------------------\n\n# loading electricity access data\npeople_without_electricity = pd.read_csv('../input/people-without-electricity/people-without-electricity-country.csv') \\\n.set_index(['Entity', 'Year']).loc['World', 2016]['Number of people without access to electricity']\n\npopulation_with_electricity = pd.read_csv('../input/people-without-electricity/share-of-the-population-with-access-to-electricity.csv')\npopulation_with_electricity['% without access'] = 100 - population_with_electricity['Access to electricity (% of population)']\n# ------------------------------------------------------\n\n# loading per capita electricity consumption data\nglobal_pc_electricity_consumption = pd.read_csv('../input/per-capita-electricity-consumption/per-capita-electricity-consumption.csv').set_index('Entity')\n# extract the most recent data for the world\nglobal_pc_electricity_consumption = global_pc_electricity_consumption[global_pc_electricity_consumption['Year']==2019].loc['World']\n# ------------------------------------------------------\n\n# loading water consumption factor data\nwater_factor = pd.read_csv('../input/water-use-embedded-in-purchased-electricity/4_water_consumption_factor_by_fuel.csv') \\\n                 .set_index('Country').loc[['United States of America', 'Canada']].stack().to_frame().rename(columns={0: 'Water Consumption Factor'})\nwater_factor.index.rename(['country', 'Fuel Type'], inplace=True)\n# ------------------------------------------------------\n\n# loading financial metrics for organizations\n# Note: Operating Income = Revenue – Cost of Sold Goods – Operating Expenses\ncdp_fin_data = pd.read_csv('../input/annual-financial-data-for-hybrid-cdp-kpi/cdp_financial_data.csv')\n# selecting only the latest year's data for a given org\ncdp_fin_data = cdp_fin_data.loc[cdp_fin_data.groupby('account_number')['Fiscal Year'].idxmax()].set_index(['organization', 'account_number'])\n# ------------------------------------------------------\n\n# loading iso country dataset for 3 digit country codes needed for plotly\niso_country = pd.read_csv('../input/country-mapping-iso-continent-region/continents2.csv')[['name', 'alpha-3']]\n# normalize country names as per water stress data\niso_country.replace({\n    'Timor-Leste': 'East Timor',\n    'Palestine, State of': 'Palestine',\n    'Holy See': 'Vatican',\n    'Eswatini': 'Swaziland',\n    'United States': 'United States of America',\n    'South Korea': 'North Korea',  # SK incorrectly mapped to North Korea \n    'Korea, Republic of': 'South Korea',  # NK incorrectly mapped to SK\n    'Tanzania': 'United Republic of Tanzania',\n    'Serbia': 'Republic of Serbia',\n    'Congo': 'Republic of the Congo',\n    'Côte D\\'Ivoire': 'Ivory Coast',\n    'Bosnia And Herzegovina': 'Bosnia and Herzegovina',\n    'Brunei Darussalam': 'Brunei',\n    'Congo (Democratic Republic Of The)': 'Democratic Republic of the Congo',\n    'Cabo Verde': 'Cape Verde',\n    'Curaçao': 'Curacao',\n    'Micronesia (Federated States of)': 'Federated States of Micronesia',\n    'Bahamas': 'The Bahamas',\n}, inplace = True)\niso_country.set_index('name', inplace=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# load corp locations dataset\ncorp_locations = pd.read_csv('../input/cdp-unlocking-climate-solutions/Supplementary Data/Locations of Corporations/NA_HQ_public_data.csv') \\\n[['organization', 'account_number', 'hq_country', 'address_city', 'address_state']].drop_duplicates()\n\n# creating an easy to match organization name to join to fortune 500 dataset\ncorp_locations['cleaned_name'] = corp_locations['organization'].str.lower() \\\n.replace(re.compile('( co\\.| corp\\.| cos\\.| llc\\.| ltd\\.| inc\\.)'), '').replace(re.compile(\"[!\\.,\\\"'&]\"), '') \\\n.replace(re.compile('( co| corp| corporation| cos| llc| ltd| inc)$'), '').str.replace('-', ' ') \\\n.replace(re.compile('^the '), '').replace(re.compile('\\s+'), ' ').str.strip()\n\ncorp_locations = corp_locations.set_index(['organization', 'account_number']).sort_index()\n# replace states with codes\ncorp_locations['address_state'] = corp_locations['address_state'].replace(STATE_CODES)\n# Renaming cities according to city names in CDP data\ncorp_locations['address_city'] = corp_locations['address_city'].replace({\n    'Cambridge': 'Cambridge, MA',\n    'San Jose': 'San José, CA',\n    'MontrÃ©al': 'Montréal',\n    'Montreal': 'Montréal',\n    'MONTREAL': 'Montréal',\n    'PALO ALTO': 'Palo Alto',\n    'PITTSBURGH': 'Pittsburgh',\n    'RESTON': 'Reston',\n    'CALGARY': 'Calgary',\n    'DENVER': 'Denver',\n    'JACKSONVILLE': 'Jacksonville',\n    'Quebec': 'Quebec City',\n    'Santa Clara,': 'Santa Clara',\n    'St. Paul': 'Saint Paul',\n    'St. Louis': 'St Louis',\n    'Saint Louis': 'St Louis',\n    'Winston Salem': 'Winston-Salem',\n})\n# Renaming address cities according to cities in CDP data - using the corresponding states\ncorp_locations['cleaned_address_city'] = corp_locations['address_city'].str.cat(corp_locations['address_state'], sep =\", \").map({\n    'Birmingham, AL': 'Birmingham, AL',\n    'Dublin, CA': 'Dublin, CA',\n    'Durham, NC': 'Durham, NC',\n    'New York, NY': 'New York City',\n    'Richmond, VA': 'Richmond, VA',\n    'Washington, DC': 'Washington, DC'\n})\n\ncorp_locations['cleaned_address_city'].fillna(corp_locations['address_city'], inplace=True)\n\n# the 'address_state' field in corp_locations contains some NaNs, but the above normalizations effectively allow us to \n# ignore the field entirely as the uniqueness of each city is preserved in the hq_country + cleaned_address_city fields\n# ------------------------------------------------------\n\n# cleaning CDP organization names to mapp with the corporate impact data\ncorp_dis = cc_dis[['country', 'account_number', 'organization']].drop_duplicates().reset_index(drop=True)\ncorp_dis['cleaned_name'] = corp_dis['organization'].str.lower() \\\n.replace(re.compile('[, ](co|corp|cos|llc|ltd|inc)\\.'), '').replace(re.compile(\"[!\\.,\\\"'&]\"), '') \\\n.replace(re.compile(' (co|corp|company|corporation|cos|plc|llc|ltd|incorporation|incorporated|inc)$'), '').str.replace('-', ' ') \\\n.replace(re.compile('^the '), '').replace(re.compile('\\s+'), ' ').str.strip()\n\n# joining CDP organizations with the impact data\nimpact_mapping = corp_dis.join(corp_env_impact, on=['country', 'cleaned_name'], how='inner')[[\n    'account_number', 'Total Environmental Cost',\n]].set_index('account_number')\n\n# cleaning column values to allow use as float\nimpact_mapping['Total Environmental Cost'] = impact_mapping['Total Environmental Cost'].str.strip('()').str.replace(',', '').astype('float32')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# HELPER FUNCTIONS\n\ndef get_cities(q, by='Row Number', year=2020, cols=['City', 'Account Number'], country=None):\n    '''Helper function to make it easier to perform queries on and return city data.'''\n\n    val2 = cities\n    if country:\n        val2 = cities[cities['Country'].isin(country)]\n    val2 = val2[(val2['Year Reported to CDP'] == year) & (val2['Question Number']==q) & (val2['Response Answer'] != 'Question not applicable')] \\\n            [cols + [by, 'Column Name', 'Response Answer']].dropna(subset=['Response Answer'])\n\n#     name = cities[(cities['Year Reported to CDP'] == year) & (cities['Question Number']==q)]['Question Name']\n#     sh = val2.shape[0]    \n#     if sh > 0:\n#         print(q, name.iloc[0], f'({sh})')\n    return val2\n\n\ndef get_climate(q, by='row_number', year=2020, cols=['organization', 'account_number']):\n    '''Helper function to make it easier to perform queries on and return climate change data.'''\n\n    val = climate[(climate['survey_year'] == year) & (climate['question_number']==q)][cols + [by, 'column_name', 'response_value']].dropna(subset=['response_value'])\n#     name = climate[(climate['survey_year'] == year) & (climate['question_number']==q)]['question_unique_reference']\n#     sh = val.shape[0]\n#     if sh > 0:\n#         print(q, name.iloc[0], f'({sh})')\n    return val\n\n\ndef get_water(q, by='row_number', year=2020, cols=['organization', 'account_number']):\n    '''Helper function to make it easier to perform queries on and return water security data.'''\n    \n    val = water[(water['survey_year'] == year) & (water['question_number']==q)][cols + [by, 'column_name', 'response_value']].dropna(subset=['response_value'])\n#     name = water[(water['survey_year'] == year) & (water['question_number']==q)]['question_unique_reference']    \n#     sh = val.shape[0]\n#     if sh > 0:\n#         print(q, name.iloc[0], f'({sh})')\n    return val\n\n\n# currency exchange rates as of 21st Nov, 2020\ncurrency_mapping = pd.DataFrame([\n    ('USD', 1),\n    ('BRL', 0.19),\n    ('EUR', 1.18),\n    ('MXN', 0.049),\n    ('GBP', 1.33),\n    ('CAD', 0.76),\n    ('COP', 0.00027),\n    ('DKK', 0.16),\n    ('PEN', 0.28),\n    ('SEK', 0.12),\n    ('AUD', 0.73),\n    ('ZAR', 0.065),\n    ('KRW', 0.0009),\n    ('CRC', 0.0016),\n    ('TRY', 0.13),\n    ('ARS', 0.012),\n    ('TWD', 0.035),\n    ('CLP', 0.0013),\n    ('IDR', 0.000071),\n    ('PHP', 0.021),\n    ('NOK', 0.11),\n    ('NZD', 0.69),\n    ('GTQ', 0.13),\n    ('CNY', 0.15),\n    ('NGN', 0.0026),\n    ('ILS', 0.3),\n    ('INR', 0.013),\n    ('MYR', 0.24),\n    ('ISK', 0.0074),\n    ('RON', 0.24),\n    ('JPY', 0.0096),\n    ('GIP', 1.3152),\n    ('SGD', 0.74),\n    ('KES', 0.0091),\n    ('BOB', 0.15),\n    ('CHF', 1.1),\n    ('THB', 0.033),\n    ('JOD', 1.41),\n    ('BDT', 0.012),\n    ('AED', 0.27),\n    ('MAD', 0.11),\n    ('TND', 0.36),\n    ('RUB', 0.013),\n    ('MDL', 0.058),\n    ('TZS', 0.00043),\n    ('BIF', 0.00052),\n    ('ETB', 0.026),\n    ('BGN', 0.61),\n    ('GHS', 0.17),\n    ('HNL', 0.041)\n], columns=['Code', 'Exchange Rate']).set_index('Code')\n\n# organizations and cities report their financial data in different currencies\n# the following dataframes are used to keep track of the currencies used by the entities and \n# their exchange rates to USD\n\ncurrency = get_cities('0.4', country=None).set_index('Account Number')[['Response Answer']] \\\n.rename(columns={'Response Answer': 'code'})\ncurrency['code'] = currency['code'].apply(lambda x: x.split()[0])\ncurrency = currency.join(currency_mapping, on='code')\n\ncurrency_cc = get_climate('C0.4').set_index(['organization', 'account_number'])[['response_value']] \\\n.rename(columns={'response_value': 'code'}).join(currency_mapping, on='code')\n\ncurrency_w = get_water('W0.4').set_index(['organization', 'account_number'])[['response_value']] \\\n.rename(columns={'response_value': 'code'}).join(currency_mapping, on='code')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<h1 style=\"color: #FFFFFF\" id=\"vulnerability-assessment\">6 Vulnerability Assessment</h1>\n\n<img src=\"https://mir-s3-cdn-cf.behance.net/project_modules/fs/191bb570799121.5baf64549e4ab.jpg\" width=480px>\n\nPhoto by [Neil V Fernando](https://www.behance.net/gallery/70799121/Editorial-Illustrations)\n<br>\n\n<h2 id=\"city-vulnerability\">6.1 City Vulnerability</h2>\n\nWe explore the theme of vulnerability by starting with the impact of climate change on cities. Global cities inevitably face a higher risk of climate hazards such as cyclones, high winds, flooding, coastal erosion and deposition and sea-level rise.[<sup>[Link]</sup>](https://journals.sagepub.com/doi/pdf/10.1177/0956247807076725) With 68% of the world's population projected to live in urban areas by 2050,[<sup>[Link]</sup>](https://www.un.org/development/desa/en/news/population/2018-revision-of-world-urbanization-prospects.html), cities face a growing responsibility to provide safe and equitable living conditions for all with sufficient opportunities for the citizens to earn their livelihoods. However, there is no denying the links between Climate Change and its impacts on the most vulnerable sections of society. These are expected to hit those\nliving in poverty the hardest, due to their more prevalent dependency on the very natural resources affected by climate change and also because they have less capacity to protect themselves, adapt or recuperate losses.[<sup>[Link]</sup>](https://www.who.int/globalchange/mediacentre/events/2011/social-dimensions-of-climate-change.pdf)\n\nCOVID-19 has already shown us how unregulated continuation of the status quo will contribute to increasing inequality. But Climate Change is far more serious problem, not to mention, a key contributor in the rise and the propagation of various infectious diseases.[<sup>[Link]</sup>](https://www.who.int/globalchange/climate/en/chapter6.pdf)\n\nThrough this section, we aim to develop metrics that track climate risks in cities and assess their ability, present and future, to tackle these risks.\n\n<h3 id=\"vulnerability-and-preparedness\">6.1.1 Vulnerability and Preparedness</h3>\n\n---\n<p id=\"#metric1\">\n<strong><em>Metric 1: Vulnerability</em></strong><br>\n<i>The susceptibility of cities to the possible risks and negative impacts of climate change, due to inadequate preparedness, high exposure, or both.</i>\n</p>\n    \n$$Vulnerability_{city} = \\sum_{i=1}^{climateHazards}max(0, ClimateExposure_{i} - Preparedness_{i})$$\n\n**Rationale**  \nA basic understanding of climate threat requires a measure of the climate exposure, that is the amount, magnitude and time-frame of the impact of risks cities will deal with in the present and the future. However, _Vulnerability_ provides a more comprehensive view, allowing cities to assess the sufficiency of their preparedness agaisnt their climate exposure. This is determined through a simple arithmetic calculation that promotes high levels of preparedness for high levels of exposure, thereby immediately encouraging planning of mitigation and adaptation measures.\n\n---"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# replacement values for mapping magnitudes, frequencies, timeframes, etc for climate hazards faced by cities\nREPLACE_CAT = {'High': 5, 'Medium High': 4, 'Medium': 3, 'Medium Low': 2, 'Low': 1, 'Do not know': 1, 'Does not currently impact the city': 0}\n\nREPLACE_CAT2 = {'Increasing': 2, 'None': 1, 'Do not know': 1, 'Decreasing': 0.5, 'Not expected to happen in the future': 0}\n\nREPLACE_TERM = {'Immediately': 4, 'Short-term (by 2025)': 3, 'Medium-term (2026-2050)': 2, 'Long-term (after 2050)': 1}","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"c2_1 = get_cities('2.1', cols=CITY_GROUP)\n\n# pivot responses on columns and group as list wherever multiple values are provided for a specific row\npiv2_1 = c2_1.pivot_table(index=CITY_GROUP+['Row Number'], columns=['Column Name'],\n                          aggfunc=lambda x: x.values[0] if len(list(x))==1 else list(x))['Response Answer']\n\n# create numerical variables using the above replacement dicts\npiv2_1['current_mag'] = piv2_1['Current magnitude of hazard'].replace(REPLACE_CAT).fillna(1)\npiv2_1['current_prob'] = piv2_1['Current probability of hazard'].replace(REPLACE_CAT).fillna(1)\n\npiv2_1['future_mag'] = piv2_1['Future expected magnitude of hazard'].replace(REPLACE_CAT).fillna(1)\npiv2_1['future_intensity'] = piv2_1['Future change in intensity'].replace(REPLACE_CAT2).fillna(1)\npiv2_1['future_freq'] = piv2_1['Future change in frequency'].replace(REPLACE_CAT2).fillna(1)\npiv2_1['time_scale'] = piv2_1['When do you first expect to experience those changes in frequency and intensity?'].replace(REPLACE_TERM).fillna(1)\n# adding a counter to keep track of the number of hazards during grouping\npiv2_1['hazard_count'] = 1\n\n# creating separate columns for category and hazard\npiv2_1[['Hazard Category', 'Hazard']] = piv2_1['Climate Hazards'].str.split(' > ', expand=True)\n\n# calculate exposure score for each risk\npiv2_1['Exposure'] = ((5 * piv2_1['current_mag'] * piv2_1['current_prob']) + \\\n                      (piv2_1['time_scale'] * piv2_1['future_mag'] * piv2_1['future_intensity'] * piv2_1['future_freq']))\n# add values for total city exposure\ncity_df = piv2_1.groupby(CITY_GROUP)[['Exposure', 'hazard_count']].sum()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The following visualization shows us that a significant proportion of the cities responding to the 2020 Cities Questionnaire face immediate risks from various Medium to High magnitude climate hazards. Proving it's essential that cities keep track, over time, of the various risks they face and moreover, continuously assess their exact level of exposure to each of them. Keeping this in mind to obtain the Vulnerability metric, we start with first calculating the _Climate Exposure_ and city _Preparedness_ below."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"temp = piv2_1[piv2_1['current_mag'].isin([4,5,3,2])].reset_index().groupby(['current_mag', 'Hazard Category'])['City'].nunique()*100/cd2020.shape[0]\n\ntemp.unstack(0).plot.barh(color=reversed(grad4), stacked=True, title='Percentage of 2020 Cities reporting medium to high magnitude hazards currently faced') \\\n.legend(['Medium Low', 'Medium', 'Medium High', 'High']);","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"---\n***Metric 1(a): Climate Exposure***  \n_Total level of risk faced by cities due to long and short term impacts of climate change._\n<br><br>\n\n$$\\begin{align}&Exposure_{city} = \\sum_{i=1}^{hazards}((5 \\times CurrentMagnitude \\times CurrentProbability) + (TimeScale \\times FutureMagnitude \\times FutureIntensity \\times FutureFrequency)), \\\\ \\\\\n\\text{where } & TimeScale \\text{ is the expected time frame for changes in frequency and intensity of climate hazards}\\end{align}$$\n<br><br>\n\n**Rationale**  \nThe lack of comparability of different climate related risks between different cities creates a gap in comprehending the level of impact on a global level. Through _Climate Exposure,_ we assign a cumulative exposure score to **each climate hazard** faced by a city, thereby allowing comparisons of risk exposure at the intra as well as intercity levels. In the calculation, the multiplication of current risks with 5 assigns a greater weight to the existing risks. This is similar to how the TimeScale is used to weight future risks. In effect, this (sub)metric would itself allow National Governments to **focus resources** towards the specific risks and cities that are most exposed to climate change, **providing opportunities** for collaboration amongst cities facing similar problems. In addition, exposure plays a key role in measuring vulnerability, as we shall see now.\n\n---\n\nWith this metric we can now visualize and compare cities on their exposure."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# distribution of cities by climate exposure\ncity_df['Exposure'].plot.hist(bins=20, figsize=(13, 7), title='Distribution of cities by total Climate Exposure', color=palette[0])\nplt.xlabel('Climate Exposure');","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(2,4,sharex=True,sharey=True,figsize=(16, 7))\ncity_df['Exposure'].hist(by='CDP Region', bins=20, ax=ax, color=palette[0]);\nfig.suptitle('Climate Exposure at the Regional Level', size=18, y=1.05)\nfig.autofmt_xdate(rotation=0)\nplt.tight_layout()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"fig = px.scatter_geo(city_df.join(city_locations.set_index('Account Number', append=True)).reset_index(), lat='lat',\n            lon='lon', \n#             hover_data=['Account Number'],\n            text='City',\n#             scope='north america',\n            size='Exposure',\n            color_continuous_scale=px.colors.sequential.Reds,            \n            color='Exposure',\n#             size_max=,\n            title='Climate Exposure - Cities around the World',\n)\nfig.update_layout(height=560, margin={\"r\":0,\"t\":30,\"l\":0,\"b\":0})\nfig.update_geos(showcountries=True, countrywidth=0.2, showcoastlines=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Since total exposure actually depends on the combined exposure for individual risks and hazards, we can compare the top 20 cities by their risk breakdowns."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# top 20 cities by exposure\nmax_exposure = city_df['Exposure'].sort_values(ascending=False)[:20]\n\n# top 20 cities climate hazard specific exposure\n# selection = alt.selection_multi(fields=['Climate Hazards'], bind='legend')\n\n# alt.Chart(piv2_1[piv2_1.index.get_level_values(1).isin(max_exposure.index.get_level_values(1))].reset_index()).mark_bar().encode(\n#     y=alt.Y('City', sort='-x'),\n#     x='sum(exposure)',\n#     color=alt.Color('category:N', scale=alt.Scale(scheme='goldred')),\n#     opacity=alt.condition(selection, alt.value(1), alt.value(0.4))\n# ).configure_axis(labelFontSize=12).properties(height=700, width=700).add_selection(selection)\n\n# text = bars.mark_text(\n#     align='left',\n#     baseline='middle',\n#     dx=3  # Nudges text to right so it doesn't appear on top of the bar\n# ).encode(\n#     text='exposure'\n# )\n# (bars + text).properties(height=900)\n\ntemp = piv2_1[piv2_1.index.get_level_values(2).isin(max_exposure.index.get_level_values(2))] \\\n.groupby(['City', 'Hazard Category'])['Exposure'].sum().unstack()\ntemp['sum'] = temp.sum(axis=1)\ntemp.sort_values('sum').drop('sum',axis=1).plot.barh(stacked=True, color=qualitative, title='Risk breakdown for cities facing maximum exposure').legend(bbox_to_anchor=(1,1));","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The exposure scores for individual hazards allow us to understand their overall impact, allowing us to determine which hazards, on average, pose the most serious risks to cities."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# cumulative exposure by climate hazard\n# selection = alt.selection_multi(fields=['category'], bind='legend')\n\n# alt.Chart(piv2_1.dropna(subset=['hazard']).reset_index()).mark_bar().encode(\n#     y=alt.Y('hazard', sort='-x'),\n#     x='sum(exposure)',\n#     color=alt.Color('category:N', scale=alt.Scale(scheme='viridis')),\n#     opacity=alt.condition(selection, alt.value(1), alt.value(0.2))\n# ).configure_axis(labelFontSize=12).properties(width=500).add_selection(selection)\n\ntemp = piv2_1.groupby(['Hazard Category', 'Hazard'])['Exposure'].mean().unstack(0)\ntemp['sum'] = temp.sum(axis=1)\ntemp.sort_values('sum').drop('sum', axis=1) \\\n.plot.barh(stacked=True, color=qualitative, figsize=(16,16),\n           title='Average Climate Exposure caused by each Natural Hazard').legend(title='Hazard Category', bbox_to_anchor=(1,1));\n# temp.sort_values().plot.barh(figsize=(13, 9), stacked=True, color=grey).legend(bbox_to_anchor=(1,1));","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"_Climate Exposure_ also enables us to obtain a more comprehensive view of the broader implications of climate change. \n\nGrouped by the social issues that go hand in hand with climate risks, we see that all areas on average see a medium to high exposure, with Increased conflict and/or crime seeing an average score close to the upper quartile of 137."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"print('Central tendency and dispersion for Exposure at the Hazard level:\\n')\nprint(piv2_1['Exposure'].describe().round(2))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# social issues arising from climate change\nsocial2_1 = piv2_1.explode('Social impact of hazard overall')\n\n# replacing all self stated categories with \"Others\"\nsocial2_1.loc[social2_1['Social impact of hazard overall'].str.contains('Other, ', na=False), 'Social impact of hazard overall'] = 'Others'\n\n# first sum exposure and count \nsocial2_1 = social2_1.groupby('Social impact of hazard overall')[['Exposure', 'hazard_count']].sum()\n\n# calcualing average exposure for every given class of social issues\nsocial2_1['avg_social_exposure'] = social2_1['Exposure']/social2_1['hazard_count']\nsocial2_1[['avg_social_exposure', 'hazard_count']].sort_values('avg_social_exposure', ascending=False).round(2)\n\n# plt.subplots(figsize=(14, 8))\n# sns.violinplot(x='exposure', y='Social impact of hazard overall',  data=social2_1.reset_index())\nsocial2_1['avg_social_exposure'].sort_values() \\\n.plot.barh(figsize=(13, 8), color= [grey]*11+ palette[:1],\n          title='Average Climate Exposure associated with each Social Issue');","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"With the next two visualizations, we try to use the exposure score to assess where the impacts of climate change will be localized, highlighting who and what exactly is likely to be the most disadvantaged due to climate risks. We do this by aggregating exposure scores by the vulnerable populations reported by cities along with various assets/services that they report will be disrupted."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# calculating level of impact on vulnerable populations \nvulner2_1 = piv2_1.explode('Please identify which vulnerable populations are affected')\n\n# replacing all self stated categories with \"Others\"\nvulner2_1.loc[vulner2_1['Please identify which vulnerable populations are affected'].str.contains('Other, ', na=False), \n              'Please identify which vulnerable populations are affected'] = 'Others'\n\n# calculating total exposure by each vulnerable population\nvulner2_1.groupby('Please identify which vulnerable populations are affected')['Exposure'].sum().sort_values() \\\n.plot.barh(figsize=(13,9), color= [grey]*8 + palette_r, xlabel='Vulnerable Population',\n           title='Exposure of vulnerable populations to climate change');","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# exposure for assets and services\nassets2_1 = piv2_1.explode('Most relevant assets / services affected overall')\n\n# replacing all self stated categories with \"Others\"\nassets2_1.loc[assets2_1['Most relevant assets / services affected overall'].str.contains('Other, ', na=False), 'Most relevant assets / services affected overall'] = 'Others'\n\nassets2_1.groupby('Most relevant assets / services affected overall')['Exposure'].sum().sort_values() \\\n.plot.barh(color= [grey]*17 + palette[:1], title='Average Exposure to Assests and Services due to climate change');","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Unsuprisingly, we see that the poorer sections of society, along with the elderly, are the most at risk and therefore the most disadvantaged due to climate change.\n\nThe average exposure to services and assets gives us an idea as to where cities must focus their adaptation and resilience-building efforts. As per the reported data, it is seen that climate change is going to have big impacts on public services such as health and sanitation, and affect people where they live, possibly resulting in mass migrations due to resource scarcity or uninhabitable conditions - both also reasons that might impact the general health of people.\n\nThe following heatmap looks at climate exposure through the coorrelated between various social issues and vulnerable populations.  We again see the elderly and the poor are disproportionately affected with a majority of the social impacts. The heatmap also highlights, once again, that increased demand for public services is more likely with unchecked climate change, confirming it is going to be one of the key social issues hurting vulnerable populations."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# finding what social impact is likely to affect which populations\ncompar2_1 = piv2_1.explode('Please identify which vulnerable populations are affected') \\\n.explode('Social impact of hazard overall')\n\n# having this value makes no sense when comparing with the vulnerable populations themselves - we want to see what those risks are\ncompar2_1 = compar2_1[compar2_1['Social impact of hazard overall']!='Increased risk to already vulnerable populations']\n\n# replacing all self stated categories with \"Others\"\ncompar2_1.loc[compar2_1['Social impact of hazard overall'].str.contains('Other, ', na=False), \n              'Social impact of hazard overall'] = 'Others'\ncompar2_1.loc[compar2_1['Please identify which vulnerable populations are affected'].str.contains('Other, ', na=False), \n              'Please identify which vulnerable populations are affected'] = 'Others'\n\nplt.subplots(figsize=(12, 8))\nsns.heatmap(compar2_1.groupby([\n    'Social impact of hazard overall', \n    'Please identify which vulnerable populations are affected'\n])['Exposure'].sum().unstack(), cmap='Reds');\n\nplt.xticks(\n    rotation=65, \n    horizontalalignment='right',\n)\nplt.xlabel('Vulnerable Populations')\nplt.title('Aggregated exposore by social issue and the populations affected')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We already know which natural hazards affect which populations. So we can perform a join between natural hazard risks and the corresponding health issues they are expected to cause (as reported by cities), and once more use _Climate Exposure_ as a means to measure the seriousness of the expected health impacts due to climate adversity, on subections of vulnerable populations.\n\nThe resulting heatmap is all red, which shows that almost every single group can expect to see above-median exposure to health issues due to climate change."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# get reported for health issues from city data\nc2_3a = get_cities('2.3a')\npiv2_3a = c2_3a[c2_3a['Column Name'].isin([\n    'Please identify which vulnerable populations are affected by these climate-related impacts',\n    'Identify the climate-related health issues faced by your city',\n    'Identify the climate hazards most significantly impacting the selected areas',\n    'Area affected by climate change',\n    'Timescale of climate-related issues for the selected health area'\n])].pivot_table(index=['City', 'Account Number', 'Row Number'], columns='Column Name', \n                aggfunc=lambda x: x.values[0] if len(list(x))==1 else list(x))['Response Answer'] \\\n.explode('Identify the climate hazards most significantly impacting the selected areas') \\\n.explode('Timescale of climate-related issues for the selected health area')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# combining health data with calculated exposure scores for climate hazards\nhealth = piv2_1[['Climate Hazards', 'Please identify which vulnerable populations are affected', 'Exposure']] \\\n.join(piv2_3a.reset_index().set_index([\n    'City', 'Account Number', 'Identify the climate hazards most significantly impacting the selected areas'\n]), on=['City', 'Account Number', 'Climate Hazards'], how='inner') \\\n.explode('Please identify which vulnerable populations are affected') \\\n.explode('Identify the climate-related health issues faced by your city').dropna(subset=['Identify the climate-related health issues faced by your city'])\n\n# replacing all self stated categories with \"Others\"\nhealth.loc[health['Please identify which vulnerable populations are affected'].str.contains('Other, ', na=False),\n          'Please identify which vulnerable populations are affected'] = 'Others'\n# dropping the same for health issues, because it corrupts the visualization\nhealth = health[~health['Identify the climate-related health issues faced by your city'].str.contains('Other, ', na=False)]\n\n# creating shorted names\nhealth['Identify the climate-related health issues faced by your city'] = health['Identify the climate-related health issues faced by your city'] \\\n.apply(lambda x: re.sub(r' \\(.*?\\)', '', x))\n\nfig, ax = plt.subplots(figsize=(12,10))\n\nsns.heatmap(health.groupby(['Please identify which vulnerable populations are affected',\n               'Identify the climate-related health issues faced by your city'])['Exposure'].mean().unstack(0), \n            cmap='Reds');\nplt.xticks(\n    rotation=65, \n    horizontalalignment='right',\n)\nplt.title('Correlations b/w health issues and vulnerable populations by average Climate Exposure')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"So we have seen how _Climate Exposure_ is a very flexible metric, useful in discussing the various impacts of climate risks. But by itself, it may not give us the complete picture of a city's _Vulnerability_ to climate change. To calculate vulnerability, which is our aim, we need to be able to evaluate the preparedness of a city to meet the expected challenges.\n\nIn question `3.0` of the 2020 Cities Questionnaire, cities are asked for responses to the actions being taken to adapt. These actions are visualized in the following heatmap along with the specific climate hazards they help mitigate.\n\nThe Word Cloud ,thereafter, sheds light on the focus areas of city efforts by plotting keywords from their action descriptions. Right off, we see a good majority of the mentions happen to be about water-related risks. This is confirmed from the heatmap where we see water-related hazards such as rainstorms and various kinds of flooding are amongst the primary reasons for considering adaptation actions. This makes sense, as cyclones, rainstorms and flooding were ranked number 1 4, and 5 respectively, in terms of their expected _Climate Exposure_. Hazard number 2 and 3, by total exposure, were extreme hot days and heatwaves, both of which too feature strongly on the heatmap, implying the requisite actions being implemented."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"c3 = get_cities('3.0', cols=CITY_GROUP)\npiv3 = c3[c3['Column Name'].isin([\n    'Sectors/areas adaptation action applies to', \n    'Action',\n    'Majority funding source', \n    'Co-benefit area',\n    'Total cost provided by the local government (currency)',\n    'Means of implementation',\n    'Action description and implementation progress',\n    'Climate hazards', 'Finance status', 'Status of action',\n    'Total cost of the project (currency)',\n    'Total cost provided by the majority funding source (currency)'\n])].pivot_table(index=CITY_GROUP+['Row Number'], columns='Column Name', \n                # if more than one value specified for a column, we aggregate as list for a given action\n                aggfunc=lambda x: x.values[0] if len(list(x))==1 else list(x))['Response Answer']\n\n# replacing all self stated categories with \"Others\"\npiv3.loc[piv3['Action'].str.contains('Other, ', na=False), 'Action'] = 'Others'\n\nfig, ax = plt.subplots(figsize=(18,13))\n\n# create heatmap for visualizing actions opposite the relevant climate hazard\nsns.heatmap(piv3.groupby('Climate hazards')['Action'].value_counts().unstack(), cmap=\"Reds\", annot=True)\n\nplt.xticks(\n    rotation=65, \n    horizontalalignment='right',\n)\nplt.title('Climate hazards and Adaptation actions taken by cities')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# visulizing descriptions for adaptation actions through a word cloud\nanswers3 = piv3['Action description and implementation progress'].dropna()\n# .apply(lambda x: ' '.join([item for item in x.lower().split() if item not in stop and len(item) > 3]))\n# answers3_3.str.split(expand=True).stack().value_counts()[:45]\n\nfig, ax = plt.subplots(figsize=(16,8))       \n\nwordcloud = WordCloud(stopwords=stop+['also'],\n                      width=1600, colormap='YlOrRd',             \n                      height=800).generate(' '.join(answers3))\n\nax.set_axis_off()\nplt.title('Word Cloud: Adaptation Action Descriptions')\nplt.imshow(wordcloud);","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"So the responses to this question provide an accurate representation of the relevant mitigative actions taken by cities to prepare for climate impact. But how would we use that to calculate _preparedness?_\n\n---\n***Metric 1(b): Preparedness***  \n_The resiliency of cities, from the sum of adaptative actions being implemented, towards the climate hazards they are exposed to._ \n<br><br>\n\n$\\begin{align} & Preparedness_{city} = \\sum_{i=1}^{actions}{Progress * FundingStatus * \\sqrt{C}}, \\\\ \n& \\text{where } C = \\text{No. of Cobenefit Areas} \\end{align}$\n<br><br>\n\n**Rationale**  \nAs mentioned above, it is imperative to gain insight into the degree of preparedness with respect to tackling climate change. Adaptation actions being implemented contribute towards a city's resiliency and provide an estimate of the cities' engagement towards mitigating climate risk. In our calculation, we assume that each action contributes as a weighted sum towards total city preparedness based on how far along it is in terms of progress, acquiring funding, and the number of additional co-benefits it might offer. _Preparedness_ too, like _Climate Exposure,_ may be calculated at the hazard level allowing cities to determine whether they are sufficiently equipped to battle the risks from particular climate events.\n\n---"},{"metadata":{"_kg_hide-output":false,"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# defining replacement dictionaries for following question on adaptation actions\nSTATUS = {'Scoping': 1, 'Pre-feasibility study/impact assessment': 2, 'Pre-implementation': 3, 'Implementation': 4,\n          'Implementation complete but not in operation': 4.5, 'Operation': 5, 'Monitoring and reporting': 5,\n          'Other, please specify: Complete': 5,\n          'Other, please specify: This is an ongoing action that is implemented as the city develops': 5,\n          'Other, please specify: Top up incentives for upgrades from fossil fuel furnaces to heat pumps underway.': 5,\n          'Other, please specify: An initial project was done in the fall of 2019 and a continuation is pending fr fall 2020': 4}\n\nFINANCE = {'Finance secured': 5, 'Feasibility finalized, and finance partially secured': 4, 'Seeking funding': 3, \n           'Feasibility undertaken': 2, 'Pre-feasibility/impact assessment study status': 1}","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# counter for tracking number of actions undertaken\npiv3['actions_count'] = 1\n\n# replace values for funding and status of action with numerical weights\npiv3['funding'] = piv3['Finance status'].replace(FINANCE).fillna(1)\npiv3 = piv3[~piv3['Status of action'].str.contains('Other, ', na=False)]\npiv3['progress'] = piv3['Status of action'].replace(STATUS).fillna(1)\n\n# counting number of cobenefit areas for specified actions, 0 if none specified\npiv3['cobenefit_count'] = piv3['Co-benefit area'].fillna('').apply(lambda x: len(x) if isinstance(x, list) else (1 if x else 0))\npiv3['cobenefit_sqrt'] = np.sqrt(piv3['cobenefit_count'])\n# calculate preparedness (TODO)\npiv3['Preparedness'] = (piv3['progress'] * piv3['funding'] * piv3['cobenefit_sqrt'])\n\n# add to city dataframe\ncity_df[['Preparedness', 'actions_count']] = piv3.groupby(CITY_GROUP)[['Preparedness', 'actions_count']].sum()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# city_df.sort_values('preparedness', ascending=False)[:30]\ncity_df.reset_index().sort_values('Preparedness', ascending=False)[:20] \\\n.plot.barh(x='City', y='Preparedness', color=palette[:2]+[grey]*18,\n           title='Top 20 cities ranked on total Preparedness', legend=False);\nplt.gca().invert_yaxis()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# comparing sector based on the actions taken for preparedness\npiv3a = piv3.explode('Sectors/areas adaptation action applies to')\npiv3a.loc[piv3a['Sectors/areas adaptation action applies to'].str.contains('Other, ', na=False), 'Sectors/areas adaptation action applies to'] = 'Other'\nsector_stack = piv3a.groupby(['City', 'Account Number', 'Sectors/areas adaptation action applies to'])['Preparedness'].sum().unstack()\nsector_stack.sum().sort_values().plot.barh(figsize=(13,9), title='How does Preparedness pan our across sectors?', color=[grey]*12 + palette_r[1:]);","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can now finally get around to calculating _Vulnerability._ In case of this case, again, the calculation is first done at the hazard level, and then aggregated at the city level. The minimum vulnerability to a hazard can only be 0 even if the level of preparedness is much higher than the exposure to that particular hazard.\n\nWe now visualize city _Vulnerability_ in the next two charts, first as the absolute value, and thereafter as it compares to _Climate Exposure._ In the second plot, the distance from the red diagonal represents the total **\"functional\" preparedness**, that is, the amount of preparedness that actually contributes to the vulnerability calculation."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# preparedness vs exposure for each climate hazard faced by a city\nhazard_stack = piv3.groupby(CITY_GROUP + ['Climate hazards'])[['Preparedness']].sum().join(piv2_1.groupby(CITY_GROUP+['Climate Hazards'])['Exposure'].sum(),\n                  on=CITY_GROUP+['Climate hazards'], how='right')\n\n# lowest vulnerability for a hazard can be 0, so replaceing negatives\nhazard_stack['vulnerability_haz'] = (hazard_stack['Exposure'] - hazard_stack['Preparedness']).apply(lambda x: 0 if x<0 else x)\n\ncity_df['Vulnerability'] = hazard_stack.reset_index().groupby(CITY_GROUP)['vulnerability_haz'].sum()\ncity_df.sort_values('Vulnerability', ascending=False, inplace=True)\n\ncity_df[:20].reset_index().plot.barh(x='City', y='Vulnerability', color=palette + [grey]*17,\n                                     title='Top 20 cities by Vulnerability', legend=False);\nplt.gca().invert_yaxis()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"city_df.sort_values('Exposure', ascending=False, inplace=True)\n\nfig, ax = plt.subplots(figsize=(19, 12),)\nax.plot([-20,2100], [-20, 2100], '-',  c=palette[0], alpha=0.8)\nax.plot([-20,2100], [-20, 1050], '-',  c=palette[2], alpha=0.6)\ncity_df.plot.scatter(x='Exposure', y='Vulnerability', color=palette[1],\n                     ax=ax, alpha=0.9, s=28, title='Vulnerability vs Climate Exposure')\n\nfor k, v in city_df[['Exposure', 'Vulnerability']][:20].iterrows():\n    ax.annotate(k[2], v, fontsize=13)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Looking at this comparison, we can tell that several highly exposed cities are doing well in terms of reducing their risk. The orange line represents a 50% or more reduction in total vulnerability through relevant actions being taken. Rio de Janeiro, although the most exposed, seems to be making a very commendable effort towards improving risk to its people. Other notable mentions are Sincelejo, Ann Arbour, Toronto, Providence, and even Johannesburg, considering the high levels of crime and disorder in the city.\n\n\n**NOTE:** We are aware that our metric on vulnerability bears some resemblance with the [ND-GAIN score](https://gain.nd.edu/assets/254377/nd_gain_technical_document_2015.pdf), however, when we began working on this metric, we had not yet come across the approach taken by ND GAIN and therefore the similarities are purely coincidental. Moreover, finding work in line with what we had set out to do just proved to us that we were on the right track!\n"},{"metadata":{},"cell_type":"markdown","source":"<h3 id=\"adaptation\">6.1.2 Adaptation</h3>\n\nOur next set of metrics looks at adaptation. In particular, how much motive exists for cities to adapt and in doing so what are the challenges they are likely to face.\n\n---\n<strong id=\"#metric2\"><em>Metric 2: Adaptation Incentive</em></strong>  \n_Motive for cities to adapt to climate change._\n<br><br>\n\n$\\begin{align} & Adaptation Incentive_{city} = \\sum_{i=1}^{climate Hazards}(Vulnerability_{i} + OpportunityPayoff_{i}), \\\\ \\\\\n  \\text{where } \\\\\n & OpportunityPayoff_{i} = \\sum_{j=1}^{action Type_{i}}(No Of Opportunites_{j} \\times {No Of Actions}_{j} \\times \\sqrt{\\sum_{k=1}^{actions_{j}} C_{k}}) \\\\ \\\\\n & (C = \\text{No. of Cobenefit Areas})\\end{align}$\n\n**Rationale**  \nWith cities responsible for [75% of global CO2 emissions](https://bit.ly/3l9uM6Y), they are also the most prone to climate change, yet [no country is doing enough](https://cnn.it/37dRFBg) to keep the earth’s temperature from rising. While high levels of vulnerability require adaptation to be a necessity. However, the opportunities associated with adaption actions, such as clean energy initiatives, new revenue streams, efficient technologies, implementation of carbon taxes provide an even greater incentive for cities to adapt to the risks posed by climate change. \n\nAs such, _Adaptation Incentive,_ calculated as a sum of the vulnerability from each climate hazard, and the expected payoff from implementing adaptation actions that enable cities to realize identified opportunities, assigns a score to the combined incentive - from risks and opportunites - for cities to adapt. Cities with a greater number of opportunities that are directly tied to their adaptation efforts are more likely to score better on opportunity payoff. In addition to the ability to realize opportunities, the opportunity payoff also considers the number of additional cobenefits they are expected to provide.\n\n---\n\nTo analyse the responses to Question `6` of the 2020 Questionnaire, we do topic modelling using [LDA (Latent Dirichlet Allocation)](https://en.wikipedia.org/wiki/Latent_Dirichlet_allocation). This classifies the responses into a set of \"topics\", each identified by a group of keywords that are closely related to the content of the corresponding topic. The topics assigned by LDA can be visualized alongside the Opportunity Category from the questionnaire. This helps us find correlations between them, which, if they exist, validate our classifications."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"topics = {}\ndef get_topic_keywords(model, count_vectorizer, n_top_words):\n    '''Adds top keywords for every topic in topics dictionary'''\n    \n    words = count_vectorizer.get_feature_names()\n    for topic_idx, topic in enumerate(model.components_):\n        topics[topic_idx+1] = []\n        for i in topic.argsort()[:-n_top_words - 1:-1]:\n            word = words[i]\n            if word != 'city':\n                topics[topic_idx+1].append(word)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"c6 = get_cities('6.0', cols=['Country', 'City', 'Account Number']) \\\n.pivot(index=['Country', 'City', 'Account Number', 'Row Number'], columns=['Column Name'])['Response Answer']\n\n# cities from the following countries have reported in Spanish\npiv6 = c6.drop(['Brazil', 'Mexico', 'Portugal', 'Costa Rica', 'Colombia', 'Peru', 'Chile', 'Argentina'])\n# removing French responses from Dijon, Nice\npiv6 = piv6.drop(('France', 'Dijon')).drop(('France', 'Nice'))\n\n# clean descriptions\ndesc = piv6['Describe how the city is maximizing this opportunity'].fillna('').map(lambda x: re.sub('[,\\.!?]', '', x.lower()))\n\n# initialise the count vectorizer with english/spanish/french (just in case) stop words\ncount_vectorizer = CountVectorizer(stop_words=stop)\n# fit and transform the processed titles\ncount_data = count_vectorizer.fit_transform(desc)\nwords = count_vectorizer.get_feature_names()\n\n# tweak the two parameters for LDA\nnumber_topics = 15\nnumber_words = 5\n# create and fit the LDA model\nlda = LDA(number_topics, n_jobs=-1, random_state=42)\n# get index corresponding max topic value for every description\npiv6['topic_id'] = lda.fit_transform(count_data).argmax(axis=1) + 1\n\n# add topic keywords to topics dict\nget_topic_keywords(lda, count_vectorizer, number_words)\n\npiv6['Topic'] = piv6['topic_id'].apply(lambda x: ', '.join(topics[x][:4]))\n\n# replacing all self stated categories with \"Others\"\npiv6.loc[piv6['Opportunity'].str.contains('Other, ', na=False), 'Opportunity'] = 'Others'\n# piv6['count'] = piv6['Describe how the city is maximizing this opportunity'].fillna('').apply(lambda x: [i for i in grid if i in x])\n\nplt.subplots(figsize=(16, 11))\n# sns.heatmap(piv6.explode('count').groupby('count')['Opportunity'].value_counts().unstack().T, annot=True);\nsns.heatmap(piv6.groupby('Topic')['Opportunity'].value_counts().unstack().T, cmap='Reds');\n\nplt.xticks(\n    rotation=65, \n    horizontalalignment='right',\n)\nplt.title('Opportunities Types and detected topics')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The heatmap confirms the existence of correlations. Looking at the topic keywords we find that not only are they closely matching the opportunity categories, which was to be expected, but they also hint at the underlying actions that go hand in hand with these opportunities. For example they bring out **\"new, projects\"**, **\"resilience, action\"**, **\"action, plan\"**, **\"solar, community\"**, **\"infrastructure, plan\"**. \n\nTo calculate _Opportunity Payoff,_ we can use the knowledge we just gained, of implicit action being present within each opportunity, to discover which adaptation actions help with which opportunities. For that, we create a mapping between adaptation actions from Question `3` of the questionnaire and the opportunities listed in question `6`."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# mapping b/w adaptation actions taken and opporunities identified by cities\n# 'Additional funding opportunities' and Increase opportunities for partnerships' not included in\n# opportunities as almost every action is compatible with realizing these\n\nclimate_action_opportunity = pd.DataFrame([\n    ('Flood mapping', 'Improved flood risk mitigation'),\n    ('Flood mapping', 'Development of climate change resiliency projects'),\n    ('Heat mapping and thermal imaging', 'Development of climate change resiliency projects'),\n    ('Heat mapping and thermal imaging', 'Reduced risk to human health'),\n    ('Landslide risk mapping', 'Development of climate change resiliency projects'),\n    ('Sea level rise modelling', 'Improved flood risk mitigation'),\n    ('Sea level rise modelling', 'Development of climate change resiliency projects'),\n    ('Biodiversity monitoring', 'Development of climate change resiliency projects'),\n    ('Biodiversity monitoring', 'Reduced risk to natural capital'),\n    ('Biodiversity monitoring', 'Development of resource conservation and management'),\n    ('Real time risk monitoring', 'Development of climate change resiliency projects'),\n    ('Crisis management including warning and evacuation systems', 'Development of climate change resiliency projects'),\n    ('Crisis management including warning and evacuation systems', 'Reduced risk to human health'),\n    ('Public preparedness (including practice exercises/drills)', 'Development of climate change resiliency projects'),\n    ('Public preparedness (including practice exercises/drills)', 'Reduced risk to human health'),\n    ('Community engagement/education', 'Reduced risk to human health'),\n    ('Projects and policies targeted at those most vulnerable', 'Improved flood risk mitigation'),\n    ('Projects and policies targeted at those most vulnerable', 'Development of climate change resiliency projects'),\n    ('Projects and policies targeted at those most vulnerable', 'Reduced risk to human health'),\n    ('Projects and policies targeted at those most vulnerable', 'Increased opportunities for investment in infrastructure projects'),\n    ('Testing/vaccination programmes for vector-borne disease', 'Reduced risk to human health'),\n    ('Disease prevention measures', 'Reduced risk to human health'),\n    ('Air quality initiatives', 'Development of climate change resiliency projects'),\n    ('Air quality initiatives', 'Reduced risk to human health'),\n    ('Incorporating climate change into long-term planning documents', 'Development of climate change resiliency projects'),\n    ('Incorporating climate change into long-term planning documents', 'Development of sustainable construction/real estate sector'),\n    ('Incorporating climate change into long-term planning documents', 'Development of sustainable transport sector'),\n    ('Restrict development in at risk areas', 'Improved efficiency of municipal operations'),\n    ('Resilience and resistance measures for buildings', 'Development of climate change resiliency projects'),\n    ('Resilience and resistance measures for buildings', 'Reduced risk to human health'),\n    ('Resilience and resistance measures for buildings', 'Increased opportunities for investment in infrastructure projects'),\n    ('Resilience and resistance measures for buildings', 'Development of sustainable construction/real estate sector'),\n    ('Hazard resistant infrastructure design and construction', 'Development of climate change resiliency projects'),\n    ('Hazard resistant infrastructure design and construction', 'Reduced risk to human health'),\n    ('Hazard resistant infrastructure design and construction', 'Increased opportunities for investment in infrastructure projects'),\n    ('Hazard resistant infrastructure design and construction', 'Development of sustainable construction/real estate sector'),\n    ('Diversifying power/energy supply', 'Development of clean energy technologies/businesses'),\n    ('Diversifying power/energy supply', 'Increased energy security'),\n    ('Diversifying power/energy supply', 'Development of energy efficiency measures and technologies'),\n    ('Economic diversification measures', 'Creation/development of carbon markets'),\n    ('Economic diversification measures', 'Carbon tax revenue'),\n    ('Flood defences – development and operation & storage', 'Improved flood risk mitigation'),\n    ('Flood defences – development and operation & storage', 'Development of climate change resiliency projects'),\n    ('Flood defences – development and operation & storage', 'Reduced risk to human health'),\n    ('Flood defences – development and operation & storage', 'Increased opportunities for investment in infrastructure projects'),\n    ('Storm water capture systems', 'Improved flood risk mitigation'),\n    ('Storm water capture systems', 'Increased water security'),\n    ('Storm water capture systems', 'Development of climate change resiliency projects'),\n    ('Storm water capture systems', 'Increased opportunities for investment in infrastructure projects'),\n    ('Storm water capture systems', 'Development of water management sector'),\n    ('Additional reservoirs and wells for water storage', 'Improved flood risk mitigation'),\n    ('Additional reservoirs and wells for water storage', 'Increased water security'),\n    ('Additional reservoirs and wells for water storage', 'Development of climate change resiliency projects'),\n    ('Additional reservoirs and wells for water storage', 'Increased opportunities for investment in infrastructure projects'),\n    ('Additional reservoirs and wells for water storage', 'Development of water management sector'),\n    ('Soil retention strategies', 'Improved flood risk mitigation'),\n    ('Soil retention strategies', 'Development of climate change resiliency projects'),\n    ('Soil retention strategies', 'Reduced risk to natural capital'),\n    ('Tree planting and/or creation of green space', 'Improved flood risk mitigation'),\n    ('Tree planting and/or creation of green space', 'Development of climate change resiliency projects'),\n    ('Tree planting and/or creation of green space', 'Reduced risk to natural capital'),\n    ('Tree planting and/or creation of green space', 'Improved efficiency of municipal operations'),\n    ('Green roofs/walls', 'Development of climate change resiliency projects'),\n    ('Green roofs/walls', 'Improved efficiency of municipal operations'),\n    ('Green roofs/walls', 'Increased opportunities for investment in infrastructure projects'),\n    ('Green roofs/walls', 'Development of sustainable construction/real estate sector'),\n    ('White roofs', 'Improved efficiency of municipal operations'),\n    ('White roofs', 'Increased opportunities for investment in infrastructure projects'),\n    ('Shading in public spaces, markets', 'Reduced risk to human health'),\n    ('Shading in public spaces, markets', 'Improved efficiency of municipal operations'),\n    ('Shading in public spaces, markets', 'Increased opportunities for investment in infrastructure projects'),\n    ('Cooling systems for critical infrastructure', 'Reduced risk to human health'),\n    ('Cooling systems for critical infrastructure', 'Increased opportunities for investment in infrastructure projects'),\n    ('Retrofit of existing buildings', 'Development of climate change resiliency projects'),\n    ('Retrofit of existing buildings', 'Increased opportunities for investment in infrastructure projects'),\n    ('Cooling centers, pools, water parks/plazas', 'Reduced risk to human health'),\n    ('Cooling centers, pools, water parks/plazas', 'Improved efficiency of municipal operations'),\n    ('Cooling centers, pools, water parks/plazas', 'Increased opportunities for investment in infrastructure projects'),\n    ('Cool pavement', 'Reduced risk to human health'),\n    ('Cool pavement', 'Improved efficiency of municipal operations'),\n    ('Cool pavement', 'Increased opportunities for investment in infrastructure projects'),\n    ('Water extraction protection', 'Increased water security'),\n    ('Water extraction protection', 'Development of resource conservation and management'),\n    ('Water extraction protection', 'Development of water management sector'),\n    ('Promoting low flow technologies', 'Increased water security'),\n    ('Promoting low flow technologies', 'Development of water management sector'),\n    ('Water butts/rainwater capture', 'Increased water security'),\n    ('Water butts/rainwater capture', 'Improved flood risk mitigation'),\n    ('Water butts/rainwater capture', 'Development of climate change resiliency projects'),\n    ('Water butts/rainwater capture', 'Development of water management sector'),\n    ('Xeriscapes – low water landscaping design', 'Increased water security'),\n    ('Xeriscapes – low water landscaping design', 'Development of resource conservation and management'),\n    ('Xeriscapes – low water landscaping design', 'Development of sustainable construction/real estate sector'),\n    ('Xeriscapes – low water landscaping design', 'Development of water management sector'),\n    ('Maintenance/repair – leaking infrastructure', 'Increased water security'),\n    ('Maintenance/repair – leaking infrastructure', 'Improved efficiency of municipal operations'),\n    ('Maintenance/repair – leaking infrastructure', 'Increased opportunities for investment in infrastructure projects'),\n    ('Maintenance/repair – leaking infrastructure', 'Development of water management sector'),\n    ('Optimizing delivery fuel mix of water supply', 'Increased water security'),\n    ('Optimizing delivery fuel mix of water supply', 'Increased opportunities for investment in infrastructure projects'),\n    ('Optimizing delivery fuel mix of water supply', 'Development of water management sector'),\n    ('Improve water supply distribution method', 'Increased water security'),\n    ('Improve water supply distribution method', 'Improved efficiency of municipal operations'),\n    ('Improve water supply distribution method', 'Increased opportunities for investment in infrastructure projects'),\n    ('Improve water supply distribution method', 'Development of water management sector'),\n    ('Promoting and incentivizing water efficiency', 'Increased water security'),\n    ('Promoting and incentivizing water efficiency', 'Improved efficiency of municipal operations'),\n    ('Water use restrictions and standards', 'Increased water security'),\n    ('Water use restrictions and standards', 'Development of resource conservation and management'),\n    ('Water efficient equipment and appliances', 'Increased water security'),\n    ('Water efficient equipment and appliances', 'Increased opportunities for investment in infrastructure projects'),\n    ('Water efficient equipment and appliances', 'Development of water management sector'),\n    ('Water smart metering', 'Increased water security'),\n    ('Water smart metering', 'Improved efficiency of municipal operations'),\n    ('Water smart metering', 'Development of water management sector'),\n    ('Water use audits', 'Increased water security'),\n    ('Water use audits', 'Improved efficiency of municipal operations'),\n    ('Water use audits', 'Development of water management sector'),\n    ('Awareness campaign/education to reduce water use', 'Increased water security'),\n    ('Awareness campaign/education to reduce water use', 'Improved efficiency of municipal operations'),\n    ('Diversification of water supply', 'Increased water security'),\n    ('Diversification of water supply', 'Development of climate change resiliency projects'),\n    ('Diversification of water supply', 'Development of water management sector'),\n    ('Increasing use of desalination', 'Increased water security'),\n    ('Increasing use of desalination', 'Development of climate change resiliency projects'),\n    ('Increasing use of desalination', 'Development of water management sector'),\n    ('Nature based solutions for water', 'Increased water security'),\n    ('Nature based solutions for water', 'Development of climate change resiliency projects'),\n    ('Nature based solutions for water', 'Development of water management sector'),\n], columns=['Action', 'Opportunities'])\n\nclimate_action_opportunity[:15].style.set_caption('Sample of the Action-Opportunity Mapping')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"With the mapping, we can now assess the contribution of the totality of adaptation actions towards realizing opportunities. This can be visualized as a simple count of the sum of opportunities for city actions."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# joing adaptation actions using action/opportunity mapping and counting available opportunities\npiv3.join(climate_action_opportunity.set_index('Action'), on='Action') \\\n.groupby(['City'])['Opportunities'].count().sort_values(ascending=False)[:20] \\\n.plot.barh(figsize=(16,11), color=palette[0:1]+[grey]*19, title='Total Opportunities linked with city Adaptation Efforts');\nplt.gca().invert_yaxis()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Thereafter, we calculate _Adaptation Incentive._ The following visualization shows the contribution of vulnerability and opportunity payoff for the top 20 cities ranked by total incentive."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# calculating the count of each kind of action and total number of associated cobenefits, joining on with corresponding opportunites\nopportunities = piv3.groupby(CITY_GROUP+ ['Climate hazards', 'Action'])[['actions_count', 'cobenefit_count']].sum() \\\n.join(climate_action_opportunity.groupby('Action')['Opportunities'].count(), on='Action')\n\n# calculate opportunity payoff\nopportunities['op_payoff'] = opportunities['Opportunities'] * opportunities['actions_count'] * np.sqrt(opportunities['cobenefit_count'])\n\n# join with vulnerability score for each climate hazard\nincentive = opportunities.groupby(CITY_GROUP+ ['Climate hazards'])[['op_payoff']].sum() \\\n.join(hazard_stack['vulnerability_haz'], on=CITY_GROUP+['Climate hazards']).reset_index().set_index(['City', 'Account Number'])\n\n# calculate adaptation incentive for each hazard using vulnerability and opportunity payoff\nincentive['incentive'] = incentive['vulnerability_haz'] + incentive['op_payoff']\n# perform final sum over hazards to get values for cities\ntemp = incentive.groupby(['City'])[['vulnerability_haz', 'op_payoff']].sum()\ntemp['incentive'] = temp.sum(axis=1)\ntemp.sort_values('incentive', ascending=False)[:20].drop('incentive', axis=1) \\\n.plot.barh(stacked=True, color=palette[:2], title='Adaptation Incentive') \\\n.legend(['Vulnerability', 'Opportunity Payoff']);\nplt.gca().invert_yaxis()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Although composed mainly of the vulnerability, this metric remains an extremely important indicator for cities that may have high opportunity payoff, but no direct incentive to adapt due to low vulnerability. As seen when we rank cities by their total opportunity payoff, the added incentive from Opportunities becomes critical to get cities to take future-oriented steps to mitigate risk from unseen climate events. Moreover, it provides the extra push needed to get city administrations to take actions that would benefit the well being of the populations living within them, even if no real risk of climate change exists."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"temp = incentive.groupby(['City'])[['op_payoff', 'vulnerability_haz']].sum()\ntemp['incentive'] = temp.sum(axis=1)\ntemp.sort_values('op_payoff', ascending=False)[:20].drop('incentive', axis=1) \\\n.plot.barh(stacked=True, color=palette_r[1:], title='Adaptation Incentive by greatest Opportunity Payoff') \\\n.legend(['Opportunity Payoff', 'Vulnerability', ]);\nplt.gca().invert_yaxis()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"So far we have discussed incentive to adapt, but circumstances for every city differ, making it more difficult, in some cases, to be able to adapt even if the incentives are there. So with our next metric we look at _Adaptability._\n\n---\n<strong id=\"#metric3\"><em>Metric 3: Adaptability</em></strong>  \n_Ability of a city to adapt to climate change._\n<br><br>\n\n$$ \\begin{align} & Adaptability_{city} = AdaptabilityFactor_{city} \\times Proactiveness_{city} \\times (10 \\times HumanCapitalScore + \\log_{10}(1 + Funding)), \\\\ \\\\ \\text{where } \\\\ & AdaptabilityFactor= \\sum_{i=1}^{factors}Impact_{i} \\times Degree_{i}, \\\\ \\text{and } & Funding = \\text{Average estimated funding available per adaptation action} \\end{align}$$\n\n**Rationale**  \nOvercoming climate change risks involves both mitigative actions, such as to reduce GHG emissions, as well as adapting to actual or future climate changes.[<sup>[Link]</sup>](https://go.nasa.gov/2KRPBaP) The adaptation ability of a city depends on the available funding and human capital, along with the total proactiveness in terms of conducting the activities needed for successful adaptation or transition. _Adaptability_ measures the ease or difficulty associated with this process of adaptation.\n\n---\n\nWe use the World Bank [Human Capital Index data](https://datacatalog.worldbank.org/dataset/human-capital-index) to arrive at a rough estimate of the human capital available to each city to plan, perform tasks, and conduct activities needed to increase city resilience and mitigate climate risk. Proactiveness is a measure of the city's drive to bring about positive change. This is estimated from the number of adaptation actions they are implementing.\n\nThe Adaptability Factor is calculated as a sum of products of the degree and impact of factors affecting adaptation efforts, as reported by cities in question `2.2`. Here the impact may be one of \"Challenges\" or \"Supports\", while the degree is the associated magnitude of the impact.\n\nThe funding available to cities is estimated from the history of implementation of adaptation actions by the city. Here we assume that the funding in the future is likely to remain in the range of the average funding for previous adaptation related actions. "},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# join adaptation action data with city currency dataframe to get exchange rates for each city\npiv3 = piv3.join(currency)\n# normalizing project costs to USD\npiv3['Total cost $'] = piv3['Exchange Rate'] * (piv3['Total cost of the project (currency)'].astype('float32'))\n\n# Calculate average funding for adaptation initiatives from question on adaptation actions\nfunding = piv3.groupby(['City', 'Account Number'])[['Total cost $', 'actions_count']].sum()\nfunding['avg_funding $'] = funding['Total cost $']/funding['actions_count']\n\n\n# create replacement dicts for factoes affecting adaptability\nDEGREES = {'Significantly challenges': 0.25, 'Moderately challenges': 0.5, 'Somewhat challenges': 1, \n           'Significantly supports': 4, 'Moderately supports': 2, 'Somewhat supports': 1}\nIMPACTS = {'Challenges': 0.5, 'Supports': 1}","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"c2_2 = get_cities('2.2', cols=CITY_GROUP)\npiv2_2 = c2_2.pivot(index=CITY_GROUP+['Row Number'], columns=['Column Name'])['Response Answer']\n\nimpact = 'Indicate if this factor either supports or challenges the ability to adapt' \ndegree = 'Level of degree to which factor challenges/supports the adaptive capacity of your city'\n\n# replace with numerical values from replacement dicts\npiv2_2[impact] = piv2_2[impact].replace(IMPACTS)\npiv2_2[degree] = piv2_2[degree].replace(DEGREES).fillna(1)\n\n# calculate adaptabilty factor\npiv2_2['adapt_factor'] = piv2_2[impact] * piv2_2[degree]\n\n# add data for avg funding for each city and for human capital index score for countries\n# inner join will drop cities for which data on adaptation actions/funding is not available\npiv2_2 = piv2_2.groupby(CITY_GROUP)[['adapt_factor']].sum() \\\n.join(funding[['avg_funding $', 'actions_count']], on=['City', 'Account Number'], how='inner') \\\n.join(human_capital['HUMAN CAPITAL INDEX 2020'], on='Country', how='inner')\n\n# TODO adaptability range\n# .join(human_capital[['HUMAN CAPITAL INDEX 2020 (LOWER BOUND)', 'HUMAN CAPITAL INDEX 2020', 'HUMAN CAPITAL INDEX 2020 (UPPER BOUND)']], on='Country', how='inner')\n\n# let's only consider cities reporting at least some funding data\npiv2_2 = piv2_2[piv2_2['avg_funding $'] > 0]\n\n# calculate adaptability\npiv2_2['Adaptability'] = (piv2_2['adapt_factor']  * piv2_2['actions_count'] * (10*piv2_2['HUMAN CAPITAL INDEX 2020'] + np.log10(1 + piv2_2['avg_funding $'])))\ncity_df['Adaptability'] = piv2_2['Adaptability']\n# .value_counts().sort_values(ascending=False)[:53].unstack() \\\n# .plot(kind='barh', cmap='tab20', stacked=True, figsize=(12, 8), title='Factors affecting adaptation ability (2020)');","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"On calculating _Adaptability,_ we see there exists a massive divide between the top 20 and bottom 20 cities in terms of how easy it may be for them to adapt. The table following the bar chart attempts to shed some light on the reason for this divide, and we see just how important the different components of _Adaptability_ are, in order for a city to score highly."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"temp = city_df[['Exposure', 'Adaptability']].dropna(subset=['Adaptability']).sort_values('Adaptability', ascending=False)\npd.concat([temp[:16], temp[-16:]]).reset_index().plot.barh(x='City', y='Adaptability', figsize=(16, 11), color=palette[1:2]*16 + palette[:1]*16,\n                                                           title='Top & bottom 16 Cities scored by Adaptability', logx=True, legend=False);\nplt.gca().invert_yaxis()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"temp2 = piv2_2.dropna(subset=['Adaptability']).sort_values('Adaptability', ascending=False)\npd.concat([temp2[:20], temp2[-20:]]).reset_index([0,1], drop=True).style.background_gradient('Reds').set_precision(2) \\\n.set_caption('Constituents of Adaptability')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"---\nThe visualisations and metrics discussed so far are a starting point for vulnerability assessment, and establish a thorough understanding of the impacts of climate change, and the needed governance. They work to expose the key issues of inequality in terms of the high climate exposure on the poor and elderly. The data further calls attention to the many risks that are often ignored, like the future strain expected on health and sanitation, law and order, food availability, education etc. \n\nThe root-level analysis undertaken to calculate _Vulnerability,_ shows the ambitious attempts of highly exposed cities to build resilience against climate hazards, and thus, their commitment towards protecting their populations. Next, we develop a methodology to determine how climate action leads to environmental and social opportunities, for example, Real-time risk monitoring (adaptation action) resulting in Development of climate change resiliency projects (opportunity), and how this opportunity payoff may provide further motivation for cities to implement forward looking adaptation actions.\n\nFinally, we assess the ability of a city to build resilience by mobilizing the economic and professional potential of its citizens. This results in bringing out the vast divide between the top and bottom cities in their ability to adapt.  \n\n\n<h2 id=\"corporate-vulnerability\">1.2 Corporate Vulnerability</h2>\n\nClimate change is a universal problem and its repercussions will be felt by many businesses. The issue has pushed many companies to start disclosing data on their climate change related financial impact, and in 2018, CDP measured these losses to be equivalent to $1 trillion. Many of these financial impacts are expected to materialize within the next five years.[<sup>[Link]</sup>](https://www.nytimes.com/2019/06/04/climate/companies-climate-change-financial-impact.html) The warming planet has made it clear that business as usual is no longer a feasible option in the long term and this is just the tip of the iceberg. Solving this problem involves getting to the root of it, and that requires us to adopt a systems approach to dealing with complex interrelationships between entities and objectives, causes and effects.\n\nWith our next metric, we aim to shed light on some of these relationships by quantifying the financial and strategic impact of climate change on corporations and comparing it with their impact on the environment and people. Finally we establish a way for CDP to find relevant social issues for corporations to address.\n\n\n---\n<strong id=\"#metric4\"><em>Metric 4: Climate Liability</em></strong>  \n_A measure of the total risk to corporations from aggregated financial and strategic impacts of climate change._\n<br><br>\n\n$$\\begin{align} & ClimateLiability_{org} = \\sum_{i=1}^{risks}(LiabilityFactor_{i} \\times TotalCost_{i}), \\\\ \\text{where } \\\\ & LiabilityFactor_{i} = Magnitude_{i} \\times Likelihood_{i} \\times Timeframe_{i} \\end{align}$$\n\n**Rationale**  \nThe risks from climate change run deeper than a simple dollar value of the total financial impacts incurred. Alongside these, various other factors such as indirect consequences on employees and stakeholders, changes in markets and preferences of customers, and effects on the social community at large must also be considered, as these in turn affect business operations. In view of this, we present a metric that, in addition to the financial impacts, uses the company's self assessment of the magnitude, likelihood and timeframe of expected climate risks, and scores the total liability to the company in terms of the extended, indirect issues.\n\n---\nThe total cost related to a climate risk is calculated as a sum of the cost of response and the direct financial impact. We then calculate liability for each risk, and aggregate values for all risks to find total _Climate Liability._"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# replacement dicts for climate change financial impact\nCREPLACE_CAT = {'High': 5, 'Medium-high': 4, 'Medium': 3, 'Medium-low': 2, 'Low': 1, 'Unknown': 1}\nCREPLACE_CAT2 = {'Virtually certain': 5, 'Very likely': 4, 'Likely': 3, 'More likely than not': 2, 'About as likely as not': 1, \n                 'Unlikely': 0.5, 'Very unlikely': 0.25, 'Exceptionally unlikely': 0.125, 'Unknown': 1}\nCREPLACE_TERM = {'Short-term': 3, 'Medium-term': 2, 'Long-term': 1, 'Unknown': 1}","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"cc2_3a = get_climate('C2.3a').pivot(index=['organization', 'account_number', 'row_number'], columns='column_name')['response_value']\n\n# replace fields with nummerical values\ncc2_3a['magnitude'] = cc2_3a['C2.3a_C9Magnitude of impact'].replace(CREPLACE_CAT).fillna(1)\ncc2_3a['likelihood'] = cc2_3a['C2.3a_C8Likelihood'].replace(CREPLACE_CAT2).fillna(1)\ncc2_3a['timeframe'] = cc2_3a['C2.3a_C7Time horizon'].replace(CREPLACE_TERM).fillna(1)\n\ncc2_3a['C2.3a_C12Potential financial impact figure â€“ minimum (currency)'] = cc2_3a['C2.3a_C12Potential financial impact figure â€“ minimum (currency)'].astype('float32').fillna(0)\n# in case a price range is provided, replace cost with minimum expected cost\ncc2_3a['C2.3a_C11Potential financial impact figure (currency)'] = cc2_3a['C2.3a_C11Potential financial impact figure (currency)'] \\\n    .astype('float32').fillna(cc2_3a['C2.3a_C12Potential financial impact figure â€“ minimum (currency)'])\n\ncc2_3a['C2.3a_C15Cost of response to risk'] = cc2_3a['C2.3a_C15Cost of response to risk'].astype('float32').fillna(0)\n\n# join with currency dataframe to get currency exchange rates\ncc2_3a = cc2_3a.join(currency_cc)\n\n# total impact = cost of response + (minimum) potential financial imapct\n# normalizing to USD\ncc2_3a['Total Financial Impact $'] = cc2_3a['Exchange Rate'] * (cc2_3a['C2.3a_C15Cost of response to risk'] + cc2_3a['C2.3a_C11Potential financial impact figure (currency)'])\n\n# calculate liability factor\ncc2_3a['liability_factor'] = cc2_3a['magnitude'] * cc2_3a['likelihood'] * cc2_3a['timeframe']\ncc2_3a['Climate Liability'] = cc2_3a['Total Financial Impact $'] * cc2_3a['liability_factor']\n\n# add all values for a company\ng2_3 = cc2_3a.groupby(['organization', 'account_number'])[['Total Financial Impact $', 'Climate Liability']].sum()\n# ignore companies with no costs reported\ng2_3 = g2_3[g2_3['Total Financial Impact $']>0]\n\n# add revenue and earnings data for organizations \ng2_3 = g2_3.join(impact_mapping, on=['account_number'], how='left').join(cdp_fin_data[['Revenue', 'Operating Income (Loss)']])\n# g2_3['cost %'] = g2_3['total_cost $']*100/g2_3['Total Environmental Cost']\n\ng2_3.reset_index().sort_values('Climate Liability', ascending=False)[:20] \\\n.plot.barh(x='organization', y='Climate Liability', color=palette[:2]+[grey]*18,\n           title='Top 20 Organizations by Climate Liability', legend=False);\nplt.gca().invert_yaxis()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now that we have figures for the total _Climate Liability,_ we can see is that the top companies by liability all happen to be large corporations. This might cause one to think _Climate Liability_ depends on the size of a company, but as we'll see next, there is only a 6% coorelation between revenue and lability.\n\nAnother interesting comparison point is between liability and the total Environmental Impact Costs of the company as provided in the HBS - Impact weighted accounts dataset. Here, we see something quite unexpected!"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"plt.subplots(figsize=(10,8))\nsns.heatmap(g2_3.drop('Total Financial Impact $', axis=1).corr(), annot=True, cmap='Reds');\nplt.xticks(rotation=45, horizontalalignment='right')\nplt.title('48% Correlation b/w Environmental Impact Cost and Climate Liability')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"temp = g2_3[['Total Environmental Cost', 'Climate Liability']].dropna().sort_values('Total Environmental Cost')\n\nd = np.polyfit(np.log10(temp['Total Environmental Cost']), np.log10(temp['Climate Liability']), 2)\nf = np.poly1d(d)\ntemp['y_pred'] = np.power(10, f(np.log10(temp['Total Environmental Cost'])))\n\nax = temp.plot.scatter(x='Total Environmental Cost', y='Climate Liability',\n                       logx=True, logy=True, color=palette[1],figsize=(16, 10));\ntemp.plot(x='Total Environmental Cost',y='y_pred', legend=False, ax=ax, color=palette[0]);\nplt.ylim([100000, 1000000000000]);","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"There is a significant correlation between the impact of organizations on the environment, and the liability they face due to climate risk. Moreover, we see there is absolutely no association between Environmental Cost and Revenue/Income, which tells us that that there are cleaner, more profitable, ways to seek high returns for shareholders.\n\nNext, we map the primary climate-related **physical risk** drivers for organizations with natural hazards (faced by cities) wherever there is a high degree of overlap or relatedness in their underlying cause. This enables us to look at the coexisting social issues corresponding to the climate risks companies face. The mapping also includes the physical risks listed in the Water Security Questionnaire, but these will be used in a later section."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# city - corporate risk mapping\nrisk_mapping = pd.DataFrame([\n    ('Extreme Precipitation > Rain storm', 'Flooding'),\n    ('Extreme Precipitation > Monsoon', 'Flooding'),\n    ('Extreme Precipitation > Rain storm', 'Changes in precipitation patterns and extreme variability in weather patterns'),\n    ('Extreme Precipitation > Monsoon', 'Changes in precipitation patterns and extreme variability in weather patterns'),\n    ('Extreme Precipitation > Heavy snow', 'Changes in precipitation patterns and extreme variability in weather patterns'),\n    ('Extreme Precipitation > Fog', 'Changes in precipitation patterns and extreme variability in weather patterns'),\n    ('Extreme Precipitation > Hail', 'Changes in precipitation patterns and extreme variability in weather patterns'),\n    ('Extreme Precipitation > Rain storm', 'Increased severity and frequency of extreme weather events such as cyclones and floods'),\n    ('Extreme Precipitation > Monsoon', 'Increased severity and frequency of extreme weather events such as cyclones and floods'),\n    ('Extreme Precipitation > Heavy snow', 'Increased severity and frequency of extreme weather events such as cyclones and floods'),\n    ('Extreme Precipitation > Fog', 'Increased severity and frequency of extreme weather events such as cyclones and floods'),\n    ('Extreme Precipitation > Hail', 'Increased severity and frequency of extreme weather events such as cyclones and floods'),\n    ('Storm and wind > Severe wind', 'Increased severity and frequency of extreme weather events such as cyclones and floods'),\n    ('Storm and wind > Tornado', 'Increased severity and frequency of extreme weather events such as cyclones and floods'),\n    ('Storm and wind > Cyclone (Hurricane / Typhoon)', 'Increased severity and frequency of extreme weather events such as cyclones and floods'),\n    ('Storm and wind > Extra tropical storm', 'Increased severity and frequency of extreme weather events such as cyclones and floods'),\n    ('Storm and wind > Tropical storm', 'Increased severity and frequency of extreme weather events such as cyclones and floods'),\n    ('Storm and wind > Storm surge', 'Increased severity and frequency of extreme weather events such as cyclones and floods'),\n    ('Storm and wind > Lightning / thunderstorm', 'Increased severity and frequency of extreme weather events such as cyclones and floods'),\n    ('Extreme cold temperature > Extreme winter conditions', 'Increased severity and frequency of extreme weather events such as cyclones and floods'),\n    ('Extreme cold temperature > Cold wave', 'Increased severity and frequency of extreme weather events such as cyclones and floods'),\n    ('Extreme cold temperature > Extreme cold days', 'Increased severity and frequency of extreme weather events such as cyclones and floods'),\n    ('Extreme hot temperature > Heat wave', 'Rising mean temperatures'),\n    ('Extreme hot temperature > Extreme hot days', 'Rising mean temperatures'),\n    ('Water Scarcity > Drought', 'Drought'),\n    ('Water Scarcity > Drought', 'Severe weather events'),\n    ('Water Scarcity > Drought', 'Increased water scarcity'),\n    ('Water Scarcity > Drought', 'Seasonal supply variability/inter annual variability'),\n    ('Wild fire > Forest fire', 'Increased likelihood and severity of wildfires'),\n    ('Wild fire > Land fire', 'Increased likelihood and severity of wildfires'),\n    ('Flood and sea level rise > Flash / surface flood', 'Rising sea levels'),\n    ('Flood and sea level rise > River flood', 'Rising sea levels'),\n    ('Flood and sea level rise > Coastal flood', 'Rising sea levels'),\n    ('Flood and sea level rise > Groundwater flood', 'Rising sea levels'),\n    ('Flood and sea level rise > Permanent inundation', 'Rising sea levels'),\n    ('Flood and sea level rise > Flash / surface flood', 'Flooding'),\n    ('Flood and sea level rise > River flood', 'Flooding'),\n    ('Flood and sea level rise > Coastal flood', 'Flooding'),\n    ('Flood and sea level rise > Groundwater flood', 'Flooding'),\n    ('Flood and sea level rise > Permanent inundation', 'Flooding'),\n    ('Chemical change > Salt water intrusion', 'Ecosystem vulnerability'),\n    ('Chemical change > Ocean acidification', 'Ecosystem vulnerability'),\n    ('Chemical change > Atmospheric CO2 concentrations', 'Ecosystem vulnerability'),\n    ('Mass movement > Landslide', ''),\n    ('Mass movement > Avalanche', ''),\n    ('Mass movement > Rock fall', ''),\n    ('Mass movement > Subsidence', ''),\n    ('Biological hazards > Water-borne disease', 'Poorly managed sanitation'),\n    ('Biological hazards > Vector-borne disease', 'Poorly managed sanitation'),\n    ('Biological hazards > Air-borne disease', ''),\n    ('Biological hazards > Insect infestation', ''),\n    ('Increased water stress', 'Increased water stress'),\n    ('Increased water stress', 'Rationing of municipal water supply'),\n    ('Increased water stress', 'Water stress'),\n    ('Increased water scarcity', 'Rationing of municipal water supply'),\n    ('Increased water scarcity', 'Increased water scarcity'),\n    ('Declining water quality', 'Declining water quality'),\n    ('Inadequate or ageing water supply infrastructure', 'Inadequate infrastructure'),\n    ('Higher water prices', 'Increased water scarcity'),\n    ('Environmental regulations', 'Ecosystem vulnerability'),\n    ('Environmental regulations', 'Declining water quality'),\n    ('Environmental regulations', 'Pollution incident'),\n    ('Increased water demand', 'Seasonal supply variability/inter annual variability'),\n    ('Drought', 'Drought'),\n    ('Drought', 'Seasonal supply variability/inter annual variability'),\n    ('Energy supply issues', 'Dependency on water intensive energy sources'),\n    ('Rationing of municipal water supply', 'Rationing of municipal water supply'),\n    ('Rationing of municipal water supply', 'Increased water scarcity'),\n    ('Rationing of municipal water supply', 'Seasonal supply variability/inter annual variability'),\n    ('Severe weather events', 'Severe weather events'),\n    ('Ecosystem vulnerability', 'Ecosystem vulnerability'),\n    ('Change in land-use', 'Change in land-use'),\n    ('Pollution incidents', 'Pollution incident'),\n    ('Pollution incidents', 'Pollution of water bodies due to fertilizer, other chemical use, or animal residues'),\n    ('Pollution incidents', 'Leaching of pollutants to groundwater bodies'),\n    ('Pollution incidents', 'Acid rock drainage and metal leaching'),\n    ('Pollution incidents', 'Increased levels of plastic in fresh water bodies'),\n    ('Pollution incidents', 'Rupture of tailings dams and toxic spills'),\n    ('Increased levels of plastic in freshwater bodies', 'Increased levels of plastic in fresh water bodies'),\n    ('Water infestation/disease', 'Poorly managed sanitation'),\n    ('Unauthorized/unregistered water connections', 'Inadequate infrastructure'),\n    ('Unauthorized/unregistered water connections', 'Increased water scarcity')\n], columns=['city_risk', 'corporate_risk'])\n\nrisk_mapping[2:24].reset_index(drop=True).style.set_caption('Sample from the risk mapping')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"With the mapping, we can now visualize correlations between the occurrence of physical risks corporations have reported and various social issues. We can also view the mean _Climate Liability,_ corresponding to each social issue to get an idea of the relationship between corporate and social impacts of climate change."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# social issues arising from climate change\nsocial_corp2_1 = piv2_1.explode('Social impact of hazard overall')\n# replacing all self stated categories with \"Others\"\nsocial_corp2_1.loc[social_corp2_1['Social impact of hazard overall'].str.contains('Other, ', na=False), 'Social impact of hazard overall'] = 'Others'\n\n# use risk mapping to combine corporate risk drivers with the number of each social issue\nsocial_risks = cc2_3a.reset_index()[['C2.3a_C3Risk type & Primary climate-related risk driver']].drop_duplicates() \\\n      .join(risk_mapping.set_index('corporate_risk'), on='C2.3a_C3Risk type & Primary climate-related risk driver', how='inner') \\\n      .join(social_corp2_1.groupby('Climate Hazards')['Social impact of hazard overall'] \\\n            .value_counts().rename('count').reset_index(1), on='city_risk', how='inner')\n\n# since the mapping is 1-to-many, to get the social issues for a single corporate risk driver, we will add up mapped values\ntemp = social_risks.groupby(['C2.3a_C3Risk type & Primary climate-related risk driver', 'Social impact of hazard overall'])['count'].sum().unstack()\n\nplt.subplots(figsize=(12, 6))\nsns.heatmap(temp, cmap='Reds');\nplt.xticks(rotation=65, horizontalalignment='right')\nplt.ylabel('Primary climate risk driver')\nplt.title('Expected co-occurrence of Social Issues with Climate Risks impacting corporations')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"social_liability = cc2_3a.groupby(['C2.3a_C3Risk type & Primary climate-related risk driver'])[['Climate Liability']].mean() \\\n      .join(risk_mapping.set_index('corporate_risk'), on='C2.3a_C3Risk type & Primary climate-related risk driver', how='inner') \\\n      .join(social_corp2_1.groupby('Climate Hazards')['Social impact of hazard overall'] \\\n            .value_counts().rename('count').reset_index(1), on='city_risk', how='inner').sort_index()\n\n# since the mapping is 1-to-many, to get the social issues for a single corporate risk driver, we will add up mapped values\ntemp = social_liability.groupby(['C2.3a_C3Risk type & Primary climate-related risk driver', 'Social impact of hazard overall'])[['Climate Liability', 'count']].sum()\n# calculating average liability linked to each social issue\ntemp['Average Liability'] = temp['Climate Liability']/temp['count']\n\nplt.subplots(figsize=(12, 6))\nsns.heatmap(temp['Average Liability'].unstack(), cmap='Reds');\nplt.xticks(rotation=65, horizontalalignment='right')\nplt.ylabel('Primary climate risk driver')\nplt.title('Mean Climate Liability associated with social issues')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"___\nAnd this brings us back to the point we were making at the start of this section. Which is, when we think linearly about cause and effect we view it as occurring within the same period displaying only obvious linkages, however, adopting systems thinking shows that cause and effect are separated in time and space. In the case of corporations, we first notice that the _Climate Liability_ they face is at least partially the result of their environmental impact from activities including, but not limited to, unregulated CO2 emissions, overconsumption of resources, poor waste management, and others that affect the health of the planet.\n\nBut in addition to that, we see an indirect association which highlights how these activites, due to their role in to various climate hazards, are closely related to and cause social issues. These issues disproportionately put the lives of the poor and vulnerable at risk, cause the loss of livelihoods, lead to mass migrations and population displacement, increase the transmission of infectious diseases, and in general affect resource availability and health. However, the world continues to view climate action by corporations as voluntary or goodwill, when in reality it should be their responsibility!\n\nIt is also of note that the CDP Climate Change Questionnaire for corporations, although comprehensive in terms of asking for climate related data, has hardly any mention of the social impacts related to corporate activities."},{"metadata":{},"cell_type":"markdown","source":"<h1 style=\"color: #FFFFFF\" id=\"climate-impact\">7 Climate Impact</h1>\n<h2 id=\"emissions\">7.1 Emissions</h2>\n\n<img src=\"https://static01.nyt.com/images/2015/11/29/opinion/sunday/29sun1/29sun1-jumbo.gif?quality=90&auto=webp\" width=800px>\n\n[NYTimes](https://www.nytimes.com/2015/11/29/opinion/sunday/what-the-paris-climate-meeting-must-do.html)\n<br><br>\n\n<h3 id=\"city-emissions\">7.1.1 City Emissions</h3>\n\nGlobal Warming will have visible effects on people's livelihoods, and a strong and adverse impact on human health. The rise in CO2 emissions, which is the primary cause of Global Warming, is projected to cause direct risks like heat waves, floods, storms, asthma and other respiratory diseases and more complex pathways of altered infectious disease patterns, disruptions of agricultural and other supportive ecosystems, and potentially population displacement and conflict over depleted resources, such as water, fertile land and fisheries.[<sup>[Link]</sup>](https://www.who.int/globalchange/GenderClimateChangeHealthfinal.pdf) \n\nIt is also known that few of the larger cities contribute disproportionately to the worldwide emissions. These include the densely populated cities in the developing world, but also highly industrialized cities in more developed nations. We can see this in the map below. Moreover, there is a strong positive correaltion between city GDP and total emissions, as seen in the scatter plot below for cities in the OECD nations."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"c4_6a = get_cities('4.6a', by='Row Name', cols=['Country', 'City', 'Account Number']).dropna(subset=['City'])\npiv4_6a = c4_6a.pivot(index=['Country', 'City', 'Account Number', 'Row Name'], columns='Column Name')['Response Answer']\n\npiv4_6a = piv4_6a[[\n    'Direct emissions (metric tonnes CO2e)',\n    'Indirect emissions from the use of grid-supplied electricity, heat, steam and/or cooling (metric tonnes CO2e)',\n    'Emissions occurring outside the city boundary as a result of in-city activities (metric tonnes CO2e)'\n]].astype('float32').groupby(['Country', 'City', 'Account Number']).sum() \\\n.join(population).join(city_locations.set_index('Account Number', append=True))\n\npiv4_6a = piv4_6a.reset_index([0])\npiv4_6a['Total Emissions'] = piv4_6a.sum(axis=1)\npiv4_6a['per_capita_emissions'] = piv4_6a['Total Emissions']/piv4_6a['Population']\npiv4_6a['log(per capita emissions)'] = np.log10(1+piv4_6a['per_capita_emissions'])\npiv4_6a['log(Total Emissions)'] = np.log10(1+piv4_6a['Total Emissions'])\n\n# piv4_6a.sort_values('per_capita_emissions', ascending=False)\n\nfig =  px.scatter_geo(piv4_6a.reset_index(), lat='lat',\n            lon='lon', \n            hover_data=['Total Emissions', 'per_capita_emissions'],\n            text='City',\n            size='log(per capita emissions)',\n            color_continuous_scale=px.colors.sequential.Reds,            \n           color ='log(Total Emissions)',\n            size_max=18,\n            title='Total (color) and per capita (size) City Emissions',\n)\nfig.update_layout(height=560, margin={\"r\":0,\"t\":30,\"l\":0,\"b\":0})\nfig.update_geos(showcountries=True, countrywidth=0.2, showcoastlines=False)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"temp = piv4_6a.join(city_gdp, on=['Country', 'City'])[['GDP_REAL_PPP', 'Total Emissions']].dropna().sort_values('GDP_REAL_PPP')\nd = np.polyfit(np.log10(temp['GDP_REAL_PPP']), temp['Total Emissions'], 1)\nf = np.poly1d(d)\ntemp['y_pred'] = f(np.log10(temp['GDP_REAL_PPP']))\n\nax = temp.plot.scatter(x='GDP_REAL_PPP', y='Total Emissions',\n                       logx=True, s=28, color=palette[1],figsize=(16, 10),\n                      title='OECD Cities - Total Emissions VS GDP');\ntemp.plot(x='GDP_REAL_PPP',y='y_pred', legend=False, ax=ax, color=palette[0]);\n\nplt.xlabel('GDP ($ PPP)');\nplt.ylabel('Total emissions (metric tonnes)');","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"To safeguard the world from the expected chaos that will ensue if Global Warming is not checked, it requires a wartime level scale of government mobilisation[<sup>[Link]</sup>](https://www.wri.org/our-work/project/world-resources-report/climate-change-adaptation-and-national-security) from cities around the world, but especially from in the developed nations who are much more equipped to take action against climate change. With this section, we dive into the next important step of climate action, _Mitigation,_ and this involves developing a metric that could help governments efficiently track the progress of their emission mitigation actions.\n\n---\n<strong id=\"#metric5\"><em>Metric 5: Lifetime Mitigation Score</em></strong>  \n_Lifetime performance (or utility) of a city's emission mitigation action at reducing direct and indirect emissions of CO2 equivalencies._  \n<br><br>\n\n$$\\begin{align} LifetimeMitigationScore & = \\log_{10}(1 + DirectEmissionReduction_{metricTonnesCO2e})\\\\ & + \\log_{10}(1 + IndirectEmissionReduction_{metricTonnesCO2e}) \\\\ & - \\log_{10}(1 + CostOfProject),\\end{align}$$\n<br>\n$\\text{where } \\\\ \\quad \\quad IndirectEmissionReduction_{metricTonnesCO2e} = 0.707 \\times (EnergySavings_{MWh} + RenewableEnergyProduced_{MWh})$\n<br><br>\n\n**Rationale**  \n\nAs can be seen in the CDP data, cities already know the amount of emissions reduced by each of the mitigation actions or projects implemented. However, there exists a need for an indicator to evaluate the actual _performance_ of each action. The _Lifetime Mitigation Score_ provides a methodology for evaluating emission reduction actions over their lifetimes based on the direct and indirect reduction in emissions. It is, therefore, to be used to judge the lifetime viability and utility of actions, and the dedication of cities in battling their climate concerns.\n\n---\nThe indirect reduction is calculated from energy savings/generation data - by approximating the amount of CO2e emissions that would otherwise take place if the specified mitigation action is not implemented. This figure is found (using the [EPA Greenhouse Gas Equivalencies Calculator](https://www.epa.gov/energy/greenhouse-gas-equivalencies-calculator)) to be 0.707 metric tonnes of CO2e per MWh of energy generation."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# get city data on emission mitigation actions\npiv5_4 = get_cities('5.4', cols=['Country', 'City', 'Account Number']) \\\n.pivot_table(index=['Country', 'City', 'Account Number', 'Row Number'], columns=['Column Name'],\n            aggfunc=lambda x: x.values[0] if len(list(x))==1 else list(x))['Response Answer']\n\n# piv5_4 = piv5_4.join(mitigation_action_opportunity.set_index('Mitigation action'), on='Mitigation action')\n# piv5_4.groupby(['City', 'Account Number'])['Opportunity'].count().sort_values(ascending=False)[:30]\n\n# selecting only the rows which are needed\npiv5_4 = piv5_4[piv5_4['Timescale of reduction / savings / energy production']=='Projected lifetime'][[\n    'Mitigation action', 'Finance status', 'Implementation status', \n    'Estimated emissions reduction (metric tonnes CO2e)', 'Energy savings (MWh)', \n    'Renewable energy production (MWh)', 'Total cost of the project', \n    'Total cost provided by the majority funding source (currency)',\n]]\n\n# converting numerical fields to float\nfor col in piv5_4.columns[3:]:\n    piv5_4[col] = piv5_4[col].astype('float32').fillna(0)\n    \n# remove rows where no data reported for anthing\npiv5_4 = piv5_4[~((piv5_4['Estimated emissions reduction (metric tonnes CO2e)']==0)\n                 &(piv5_4['Energy savings (MWh)']==0)\n                 &(piv5_4['Renewable energy production (MWh)']==0))]\n    \n# join with currency dataframe to get currency exchange rates\npiv5_4 = piv5_4.join(currency)\n\n# converting cost to USD\npiv5_4['total cost $'] = piv5_4['Exchange Rate'] * piv5_4['Total cost of the project']\n\n# calculate action score for emission rediction\n# 1 MWh => 0.707 metric tonnes CO2e\npiv5_4['action_score'] = (np.log10(1 + (0.707 * (piv5_4['Energy savings (MWh)'] + piv5_4['Renewable energy production (MWh)']))) + \\\n                          np.log10(1 + piv5_4['Estimated emissions reduction (metric tonnes CO2e)']) - \n                          np.log10(1 + piv5_4['total cost $']))\n\npiv5_4['action_count'] = 1\nax = piv5_4['action_score'].plot.hist(bins=20, color=palette[0],\n                                 title='Distribution of Emission Reduction Actions by Mitigation Score', figsize=(10,7));\nplt.xlabel('Mitigation Score');\n# piv5_4.reset_index(inplace=True)\n# fig, ax = plt.subplots(figsize=(18, 12)) \n# sns.violinplot(ax=ax, x=piv5_4[\"CDP Region\"], y=piv5_4[\"action_score\"], palette='Reds');","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The cumulative average score for the entire set of actions implemented by cities gives us some insight into the efficiency of cities at mitigating emissions. Here, we see how the top scoring cities compare with those that make up the tail end with respect to the _Mitigation Score._"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"actions = piv5_4.groupby(['Country', 'City', 'Account Number'])[['action_score', 'action_count']].sum()\n# calculate average action score\nactions['Mitigation Score'] = actions['action_score']/actions['action_count']\n\n# top 20 cities with at least 2 mitigation actions implemented - by average action score\ntemp = actions[actions['action_count']>1].reset_index().sort_values('Mitigation Score', ascending=False)\n\npd.concat([temp[:10], temp[-10:]]).plot.barh(x='City', y='Mitigation Score', color=palette[1:2]*10 + palette[:1]*10,\n           title='Top & bottom 10 cities by average Mitigation Score', legend=False);\nplt.gca().invert_yaxis()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The top half of this chart is primarily populated with cities that happen to be in more developed nations, such as USA, UK, Japan, and Denmark. This shows that it is significantly easier for cities in the developed parts of the world to plan and implement emission reduction projects which perform better over their lifetimes, and therfore the onus should first be on these cities to reduce their carbon footprints.\n\n---\n\n<h3 id=\"corporate-emissions\">7.1.2 Corporate Emissions</h3>\n\nThe Corporate Vulnerability section highlighted the negative impact of corporate action on the environment. Diving deeper into these impacts, with the visualization below, we find that the Fossil Fuel and Power Generation industries are, on average, responsible for the greatest amount of CO2e emissions. Given that corporations are accountable for a significant proportion of total worldwide GHG emissions, the responsibility to mitigate climate change and address the social inequality caused by climate hazards should not fall on the shoulders of governments alone."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"level = 'primary_industry'\n# create datarames to store emission data by industry and for organizations\ncorps_frame = cc_dis.groupby(level)['organization'].count().to_frame()\n# emissions_frame = cc_dis.groupby(['organization', 'account_number'])[['primary_sector']].count()\n\n# query scope 1 emission data and calculate mean by industry\ncc6_1 = get_climate('C6.1', by='row_name', cols=['country', 'organization',  'account_number', 'primary_industry', 'primary_sector']) \\\n.pivot(index=['organization', 'account_number', 'primary_industry', 'primary_sector', 'row_name'], columns='column_name')['response_value']\ncc6_1['C6.1_C1Gross global Scope 1 emissions (metric tons CO2e)'] = cc6_1['C6.1_C1Gross global Scope 1 emissions (metric tons CO2e)'].astype('float32')\n\n# emissions_frame['Scope 1'] = cc6_1.groupby(['organization', 'account_number'])['C6.1_C1Gross global Scope 1 emissions (metric tons CO2e)'].sum()\ncorps_frame['Scope 1'] = cc6_1.groupby(level)['C6.1_C1Gross global Scope 1 emissions (metric tons CO2e)'].mean()\n\ncc6_3 = get_climate('C6.3', by='row_name', cols=['country', 'organization',  'account_number', 'primary_industry', 'primary_sector']) \\\n.pivot(index=['organization', 'account_number', 'primary_industry', 'primary_sector', 'row_name'], columns='column_name')['response_value']\n\n# query scope 3 emission data and calculate mean by industry\ncc6_3['C6.3_C1Scope 2, location-based'] = cc6_3['C6.3_C1Scope 2, location-based'].astype('float32')\ncc6_3['C6.3_C2Scope 2, market-based (if applicable)'] = cc6_3['C6.3_C2Scope 2, market-based (if applicable)'].astype('float32')\ncc6_3['sum'] = cc6_3['C6.3_C1Scope 2, location-based'] + cc6_3['C6.3_C2Scope 2, market-based (if applicable)']\n\n# emissions_frame['Scope 2'] = cc6_3.groupby(['organization', 'account_number'])['sum'].sum()\ncorps_frame['Scope 2'] = cc6_3.groupby(level)['sum'].mean()\n\n# query scope 3 emission data and calculate mean by industry\ncc6_5 = get_climate(\"C6.5\", by='row_name', cols=['country', 'organization',  'account_number', 'primary_industry', 'primary_sector']) \\\n.pivot(index=['organization', 'account_number', 'primary_industry', 'primary_sector', 'row_name'], columns='column_name')['response_value']['C6.5_C2Metric tonnes CO2e']\ncc6_5 = cc6_5.astype('float32')\n\n# emissions_frame['Scope 3'] = cc6_5.groupby(['organization', 'account_number']).sum()\ncorps_frame['Scope 3'] = cc6_5.groupby(level).mean()\n\n# emissions_frame = emissions_frame[['Scope 1', 'Scope 2', 'Scope 3']]\n# emissions_frame['Total Emissions'] = emissions_frame.sum(axis=1)\n# plot mean emission data by indusrty\ncorps_frame.reset_index() \\\n.plot.barh(x=level, y=['Scope 1', 'Scope 2', 'Scope 3'], figsize=(16,16), color=palette,\n           title='Average Emissions by Industry',logx=True, xlabel='Industry');\nplt.xlabel('Metric tonnes of CO2e (logarithmic)')\nplt.gca().invert_yaxis()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Positive steps taken by corporations to reduce their emissions can play a huge role in meeting the central aim of the Paris Agreement - limiting global temperature rise to below 2 degrees celcius this century. In this section, we provide corporates with a measure to track the efficacy of their emission reduction initiatives in order to facilitate actions that are more efficient and impactful.\n\n---\n<strong id=\"#metric6\"><em>Metric 6: Reduction Initiative Score</em></strong>  \n_The utility of a corporate emission reduction initiative at reducing CO2 equivalent emissions._  \n<br><br>\n\n$\\begin{align} ReductionInitiativeScore_{lifetime} & = \\log_{10}(1 + AnnualCO_{2}eSavings \\times Lifetime) \\\\ & + \\log_{10}(1 + AnnualMonetarySavings \\times Lifetime) \\\\ & - \\log_{10}(1 + TotalLifetimeInvestment)\\end{align}$\n<br><br>\n\n$\\begin{align} ReductionInitiativeScore_{annual} & = \\log_{10}(1 + AnnualCO_{2}eSavings) \\\\ & + \\log_{10}(1 + AnnualMonetarySavings) \\\\ & - \\log_{10}(1 + TotalLifetimeInvestment/Lifetime), \\\\ \\\\ \\text{where } \\\\ & Lifetime \\text{ is a score between 0 - 30 based on reported lifetime of initiative} \\end{align}$\n<br><br>\n\n**Rationale**  \nThe _Reduction Initiative Score_ performs cost-benefit analysis on the emission reduction initiatives proposed/implemented by corporations. By assessing initiative utility at the lifelime as well as annual periods, it allows for a direct comparison between the immediate and medium/long-term gains expected from implementing the initiatives.\n\n---"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# create replacement dict for scoring reported lifetimes\nLIFETIME = {'<1 year': 0.5, '1-2 years': 1, '3-5 years': 4, '6-10 years': 8, 'Ongoing': 8,\n            '11-15 years': 13, '16-20 years': 18, '21-30 years': 25, '>30 years': 30}","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# query corporate data on reduction initiatives\npiv4_3b = get_climate('C4.3b').pivot(index=['organization', 'account_number', 'row_number'], columns='column_name')['response_value']\n\ncolumns = [\n    'C4.3b_C1Initiative category & Initiative type_G',\n    'C4.3b_C1Initiative category & Initiative type',\n    'C4.3b_C8Estimated lifetime of the initiative',\n    'C4.3b_C2Estimated annual CO2e savings (metric tonnes CO2e)',\n    'C4.3b_C5Annual monetary savings (unit currency â€“ as specified in C0.4)',\n    'C4.3b_C6Investment required (unit currency â€“ as specified in C0.4)',\n]\n# selecting only relevant data to metric calculation\ncc4_3b = piv4_3b[columns]\n# replacing self defined initiative categories with \"Others\"\ncc4_3b.loc[cc4_3b['C4.3b_C1Initiative category & Initiative type'].str.contains('Other, ', na=False), 'C4.3b_C1Initiative category & Initiative type'] = 'Others'\n\n# converting numerical fields to float\nfor col in cc4_3b.columns[-3:]:\n    cc4_3b[col] = cc4_3b[col].astype('float32')\n\n# join with currency dataframe to get currency exchange rates\ncc4_3b = cc4_3b.join(currency_cc)\n\n# converting cost to USD\ncc4_3b['savings $'] = cc4_3b['Exchange Rate'] * cc4_3b['C4.3b_C5Annual monetary savings (unit currency â€“ as specified in C0.4)']\ncc4_3b['investment $'] = cc4_3b['Exchange Rate'] * cc4_3b['C4.3b_C6Investment required (unit currency â€“ as specified in C0.4)']\n\n# assigning lifetime scores based on replacement dict\ncc4_3b['lifetime'] = cc4_3b['C4.3b_C8Estimated lifetime of the initiative'].replace(LIFETIME)\n\n# calculating rediction initiative - lifetime - score\ncc4_3b['Lifetime Score'] = np.log10(1+cc4_3b['C4.3b_C2Estimated annual CO2e savings (metric tonnes CO2e)'] * cc4_3b['lifetime']) + \\\n                           np.log10(1+ cc4_3b['savings $'] * cc4_3b['lifetime']) - \\\n                           np.log10(1+cc4_3b['investment $'])\n\n# calculating rediction initiative - annual - score\ncc4_3b['Annual Score'] = np.log10(1+cc4_3b['C4.3b_C2Estimated annual CO2e savings (metric tonnes CO2e)']) + \\\n                         np.log10(1+ cc4_3b['savings $']) - \\\n                         np.log10(1+cc4_3b['investment $']/cc4_3b['lifetime'])\n\ncc4_3b['initiative_count'] = 1\n# cc4_3b.sort_values('lifetime_score', ascending=False)[:40]\nfig, ax = plt.subplots(1,2, figsize=(15,5), sharey=True)\ncc4_3b.hist('Lifetime Score', ax=ax[0], color=palette[0], bins=20)\ncc4_3b.hist('Annual Score', ax=ax[1], color=palette[1], bins=20);\nplt.tight_layout()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The performance of organizations at implementing cost-effective, impactful initiatives can be measured by averaging the scores received on the entirety of their initiatives. The following visualization compares annual and lifetime scores for some of the highest scoring companies. Looking at the corporations that comprise the top 20 by their initiative scores, it is found that a majority are in the energy and transportation industries, with high individual CO2e emissions. This is a positive finding, as it shows that some of the large corporations with massive carbon footprints are taking the right steps to be more climate friendly, but it also lends to the fact that the large emitters have a greater potential for implementing actions to counteract Global Warming."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# calculating mean scores at the organization level\ninitiatives = cc4_3b.groupby(['organization', 'account_number'])[['Lifetime Score', 'Annual Score']].mean()\ninitiatives.columns = ['Avg Lifetime Score', 'Avg Annual Score']\ninitiatives['initiative_count'] = cc4_3b.groupby(['organization', 'account_number'])['initiative_count'].sum()\n\n# top 20 organizations by avg lifetime scores on their initiatives\ninitiatives.reset_index().sort_values('Avg Lifetime Score', ascending=False)[:20] \\\n.plot.barh(x='organization', y=['Avg Lifetime Score', 'Avg Annual Score'], figsize=(16,12), color=palette[:2],\n          title='Average Scores for corporate Emission Reduction Initiatives');\nplt.gca().invert_yaxis()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n\nMoreover, we can find the exact kinds of initiatives that perform best, by grouping on the category of the initiative. Here we see that Emission Reduction during Non-energy industrial processes provides the best results in terms of having about average costs and high annual CO2e savings."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"initiative_cat = cc4_3b.groupby(['C4.3b_C1Initiative category & Initiative type_G']) \\\n[columns+['lifetime', 'Annual Score', 'Lifetime Score']].mean().rename_axis('Initiative category')\n\ninitiative_cat.columns = ['Avg annual CO2e savings', 'Avg annual monetary savings $', 'Avg lifetime investment $', 'Avg lifetime', 'Avg Annual Score', 'Avg Lifetime Score']\n\n# initiative_cat['Total initiatives'] = cc4_3b.groupby(['C4.3b_C1Initiative category & Initiative type_G'])['initiative_count'].sum()\ninitiative_cat.round(2).sort_values('Avg Lifetime Score', ascending=False).style.background_gradient('Reds').set_precision(2) \\\n.set_caption('Comparing Initiatives by Category')\n\n# sns.violinplot(data=cc4_3b, x='lifetime_score', y='C4.3b_C1Initiative category & Initiative type_G')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"---\nMitigation of climate change is a crucial step for cities to prosper, but on viewing cities that perform best on their mitigation actions, we notice how developed cities score better. This finding hints at the underlying disparity in the availability of technologies and resources to invest in climate action, and is a reminder of how populations of countries that have contributed the least to global warming are the most vulnerable to the death and diseases brought about by higher temperatures while being the least equipped to tackle them.[<sup>[Link]</sup>](https://www.un.org/en/chronicle/article/health-effects-global-warming-developing-countries-are-most-vulnerable)\n\nFurther, tackling climate change can no longer be voluntary for corporates when the emissions they cause play an equally important role in the problem. This is especially striking when you consider that _only_ 100 of all the hundreds of thousands of companies in the world have been responsible for 71% of the global GHG emissions since 1998.[<sup>[Link]</sup>](https://www.activesustainability.com/climate-change/100-companies-responsible-71-ghg-emissions/) \n\nThis recurring theme of inequality in different contexts underlines the need for the large emitters to take bold and immediate action. Applying metrics such as the Lifetime Mitigation and Reduction Initiative scores allows both, cities and corporations, to assess their emission reduction strategies and thereby enable themselves to better counteract the social affects of global warming.\n\n---\n\n<h2 id=\"energy\">7.2 Energy</h2>\n\n<img src=\"https://i.guim.co.uk/img/media/00599b646cc4bf49445f1c8cc2ceb886109cd291/0_0_2560_1536/master/2560.jpg?width=1020&quality=85&auto=format&fit=max&s=e257bb67e7de4462155ae2e654d98c06\" width=800><br>\n\nIllustration by [Sebastien Thibault](https://www.behance.net/sebastienthibault)\n<br>\n\n<h3 id=\"renewable-energy\">7.2.1 Renewable Energy</h3>\n\nIn the previous section, we saw that the energy sector plays an important role in the severity of climate change due to high levels of GHG emissions. Using more renewable energy can lower the prices of and demand for natural gas and coal by increasing competition and diversifying our energy supplies.[<sup>[Link]</sup>](https://www.ucsusa.org/resources/benefits-renewable-energy-use)\n\nOver the last few years, the greater levels of awareness, cheaper technologies, and concerns regarding our fast heating planet, have resulted in an increasing shift towards renewables. Plotting the data reported by cities provides an idea about the distribution of renewable energy around the world. It is seen that renewable energy capacity is directly correlated with the level of development of the city, existing in orders of magnitude higher quantities in cities in the more developed nations/continents. This intra-city level inequality shows how developing countries will suffer strong climate change impacts due to the lack of technological advances required to increase their transition efforts resilience."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# query city responses on renewable energy capacity (for 2020)\npiv8_4 = get_cities('8.4', by='Row Name', cols=CITY_GROUP, country=None) \\\n.pivot(index=CITY_GROUP + ['Row Name'], columns=['Column Name'])['Response Answer']\n\npiv8_4 = piv8_4.reset_index(4)\npiv8_4['Row Name'].replace({'Other, please specify': 'Others'}, inplace=True)\n\n# convert column to float \npiv8_4['MW capacity'] = piv8_4['MW capacity'].astype('float32').fillna(0)\n\npiv8_4.groupby(['CDP Region', 'Row Name'])['MW capacity'].mean().unstack() \\\n.plot.barh(figsize=(12,12), logx=True, color=[qualitative[i] for i in [1, -1, 3, 0, -5, -3]],\n           title='Average Renewable Energy capacity (MW) in Cities - by Region') \\\n.legend(title='Renewable Energy Source', bbox_to_anchor=(1.35,1));\nplt.gca().invert_yaxis()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"However, the requirements of cities differ depending on their populations. To get a better understanding of this, we plot city renewable energy capacities alongside city populations, and we see that a majority of the cities having higher than average populations do not even match the _median_ levels of renewable energy capacity."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# join with per capita energy consumption\ng8_4 = piv8_4.groupby(['Country', 'City', 'Account Number'])[['MW capacity']].sum() \\\n.join(energy_use, on='Country', how='inner').join(population, on=['City', 'Account Number']) \\\n\n# drop cities with no renewable energy installed\ng8_4 = g8_4[g8_4['MW capacity']>0]\n\n# calculate total ciry energy use in MWh\ng8_4['estimated city energy use MWh'] = g8_4['Population'] * g8_4['Energy consumption per capita (kWh)']/1000 \n\n# calculate per capita MW capacity\ng8_4['MW pc'] = g8_4['MW capacity']/g8_4['Population']\n\n# estimating no. of years from no. of hours in a year (24 * 365) = 8760\ng8_4['Demand'] = g8_4['estimated city energy use MWh']/(g8_4['MW capacity'] * 8760)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"g8_4.sort_values('Demand', ascending=False, inplace=True)\n\nfig, ax = plt.subplots(figsize=(21, 12))\ng8_4.plot.scatter(x='Population', y='MW capacity', color=palette[0],\n                     ax=ax, alpha=0.7, s=30, title='Population VS Renewable Energy Capacity (MW)')\n\nmeanMW = g8_4['MW capacity'].median()\nmeanPop = g8_4['Population'].mean()\nax.plot([0,100000000], [meanMW, meanMW], '-',  c=palette[1])\nax.plot([meanPop, meanPop], [0,100000000], '-',  c=palette[2])\n\nplt.xscale('log')\nplt.yscale('log')\nplt.grid()\nplt.xlim([100, 100000000])\nplt.ylim([0.001, 100000000])\n\nax.annotate('Median Renewable Energy Capacity (MW)', (500, 50), fontsize=12, fontweight=550)\nax.annotate('Mean City Population', (1400000, 0.005), fontsize=12, fontweight=550)\nfor k, v in pd.concat([g8_4[:20], g8_4[-20:]])[['Population', 'MW capacity']].iterrows():\n    ax.annotate(k[1], v, fontsize=11)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"A way to measure this inadequacy of renewables in cities would be to find a way to compare city capacity the total energy demand, and that brings us to our next metric.\n\n---\n<strong id=\"#metric7\"><em>Metric 7: Renewable Energy Public Demand</em></strong>  \n_Years of renewable energy generation required, at the current capacity, to meet annual energy demand for a city._\n<br><br>\n\n$\\begin{align} & Demand_{years} = \\frac{TotalEnergyUse_{MWh}}{RenewableEnergyCapacity_{MW} \\times 8760}, \\\\  \\\\\n \\text{where } \\\\ & TotalEnergyUse_{MWh} = Population_{city} \\times PerCapitaEnergyConsumption_{MWh} \\\\ \\\\ \n (8760 & = \\text{Number of hours in a year}) \\end{align}$\n<br><br>\n\n**Rationale**  \nA good methodology for cities to evaluate their renewable energy capacity is to calculate how sufficient it would be in, hypothetically, meeting the city's total energy requirements. This metric provides a way to do so by calculating the number of years of operation, at current capacity, required to fulfil the citizenry's annual energy demand.\n\n---\n\nIt must be noted that the above calculation is based on the assumption that the per capita energy consumption in cities is the same as the national average. Although, in general, the energy consumption is cities is likely to be higher than rural and suburban areas, [in some cases](https://core.ac.uk/download/pdf/7043741.pdf) this might even be reversed. Due to lack of city-level data for energy consumption, we were forced to work with the national average, however, cities may easily rectify this by substituting the corresponsing city level values in the calculation.\n\nCities with the lowest performance on this metric have been plotted in the next bar chart. It really says something about city management when your renewable capacity is so low that you need over 120,000 years of it in a given year..."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"g8_4[:30].reset_index().plot.barh(x='City', y='Demand', figsize=(16,12), legend=False, color=palette[:1] + [grey]*29,\n                                 title='No. of years of Power Needed from Current Renewable Capacity to meet Annual City Demand');\nplt.gca().invert_yaxis()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(21, 12))\ng8_4.plot.scatter(x='Demand', y='MW capacity', color=palette[1],\n                     ax=ax, alpha=0.9, s=30, title='Years of demand VS Current Renewable Energy Capacity (MW)')\nplt.xscale('log')\nplt.yscale('log')\nplt.grid()\n\nfor k, v in pd.concat([g8_4[:20], g8_4[-20:]])[['Demand', 'MW capacity']].iterrows():\n    ax.annotate(k[1], v, fontsize=11)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Comparing energy capacity with the years of energy generation needed to meet annual demands shows an inverse relationship, as expected. However, that the 20 cities at both extremes, by _Demand,_ overlap almost entirely with the extremes by Energy Capacity is an interesting finding as it shows that total energy consumption in cities is **roughly** standard, irrespective of location or population. We see this is true with the following distribution of cities by their estimated total energy consumption, which further implies that cities with lower renewable energy capacities end up sourcing even more of their energy from fossil fuels."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"g8_4['estimated city energy use MWh'].plot.hist(bins=160, figsize=(18, 6), color=palette[0],\n                                                title='Distribution of cities by Energy Consumption');\nplt.xlabel('Total Energy Consumption (MWh)');","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now for the lowest scoring cities (that is with the greatest _Demand),_ we can once more use the GHG emissions calculator and estimate the amount of _additional_ CO2e emissions that could be saved were the city to ramp up its total Renewable Energy capacity to the _median_ of the reported data. \n\nThis calculation assumes that the new sources of energy will be functional year-round, and total energy requirements won't exceed current levels. We have also left out New York City from this, as it already has above median capacity."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# calculate additional power generation that becomes possible\ng8_4['MWh diff'] = (g8_4['MW capacity'].median() - g8_4['MW capacity'])*8760\n# calculate whether new capacity overshoots total city demand, in which case select the amount needed to satisfy remaining \n# demand after considering existing capacity\ng8_4['New Power'] = g8_4.apply(lambda x: x['MWh diff'] if (x['MWh diff'] + x['MW capacity']*8760)<x['estimated city energy use MWh'] else x['estimated city energy use MWh']-x['MW capacity']*8760, axis=1)\n# once more estimating 0.707 metric tonnes per MWh of energy generation\ng8_4['Emissions Saved'] = g8_4['New Power'] * 0.707\n\n# select cities with highest demand and then sort on emissions saved\ng8_4.drop('New York City', level=1)[:30].sort_values('Emissions Saved').reset_index() \\\n.plot.barh(x='City', y='Emissions Saved', figsize=(16, 12), legend=False, color=[grey]*16 + palette_r[1:]*7,\n           title='Estimated CO2e Emissions (metric tonnes) saved by ramping up Renewable Energy Capacity');","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The implications of the results are obvious. A straightforward increase of renewable capacity to 40 MW results in annual emission reductions of approximately 200,000 metric tonnes for each city!\n\nThis shows that transition to Renewable Energy has the potential to equip cities with a wide range of opportunities that improve economic, social and environmental conditions. That being said, we must understand the reasons behind cities having low renewable energy. Lack of resources, funding, and technology, in poorer cities, results in them being unable install adequate renewable sources, and instead cities turn to cheap fuels to help with economic growth. But climate change is a global issue and such scenarios require a call for action where the developed countries, whose past emissions have led them to where they are, lend a hand in making renewable energy accessible and available for all. \n\nNext, let's look at the role of corporations in energy consumption.\n\n\n<h3 id=\"corporate-energy-consumption\">7.2.2 Corporate Energy Consumption</h3>\n\nAccess to energy is fundamental to improving the quality of life and is key for economic development. With the world's demand for electricity set to grow by 85 per cent by 2040 - this increase is more than today's total use of electricity in the USA, the European Union, China, Russia, Japan, Australia and India.[<sup>[Link]</sup>](https://www.theworldcounts.com/challenges/climate-change/energy/global-electricity-consumption/story) An important question we must ask is how much of this electricity use is absolutely necessary, and how much simply the result of availability and lack of awareness. This is because a unique kind of inequality plagues the world, called Energy Poverty, which is the lack of access to modern energy services. \n\nEnergy poor households experience inadequate levels of essential energy services due to a combination of reasons, such as high energy expenditure, low household incomes, inefficient buildings and appliances, and specific household energy needs. It is estimated that more than 50 million households in the European Union alone face Energy Poverty.[<sup>[Link]</sup>](https://www.energypoverty.eu/about/what-energy-poverty)\n\nWith this section, we apply the inequality lens to understand the issue of corporate energy overconsumption and the impacts of their fuel use. We start with electricity consumption, and with the next two visualization understand the gap we are trying to address."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# visualizing people without electricity data\nfig = px.choropleth(population_with_electricity, locations=\"Code\",\n        color=\"% without access\",\n        hover_name=\"Entity\",\n#         projection=\"natural earth\",\n        animation_frame='Year',\n#         projection='orthographic',\n        color_continuous_scale=px.colors.sequential.OrRd,            \n        title='People without electricity (% by country)',\n)\nfig.update_layout(height=550, margin={\"r\":0,\"t\":30,\"l\":0,\"b\":0})\nfig.update_geos(resolution=110, showcountries=True, countrywidth=0.2, \n                showframe=False, showcoastlines=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"total = 'C8.2a_C4Total (renewable and non-renewable) MWh'\n\n# load corporate energy use data\ncc8_2a = get_climate('C8.2a', by='row_name', cols=['country', 'organization', 'account_number', 'primary_industry'])\n\n# selecting rows relevant to electricity consumption and joining with organization revenue, earnings\ncc8_2a = cc8_2a[cc8_2a['row_name'] == 'Consumption of purchased or acquired electricity'] \\\n.pivot(index=['country', 'primary_industry', 'organization', 'account_number', 'row_name'], columns='column_name')['response_value'] \\\n.dropna(subset=[total]) #.join(cdp_fin_data[['Revenue', 'Operating Income (Loss)']])\n\n# converting to float and dropping corporations with no reported electricity consumption  \ncc8_2a[total] = cc8_2a[total].astype('float32')\n\n# adding to corps_frame for industry level visualization\ncorps_frame['Electricity Consumption (MWh)'] = cc8_2a.groupby('primary_industry')[total].mean()\n\ncorps_frame.reset_index().sort_values('Electricity Consumption (MWh)') \\\n.plot.barh(x='primary_industry', y='Electricity Consumption (MWh)', legend=False, color=[grey]*11+palette_r[1:],\n           title='Average Electricity Consumption by Industry (in MWh)', xlabel='Industry'),;","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"For context, the average human being requires only about 3500 kWh of electricity in an entire year. That is whopping 7 orders of magnitudes lower than the average corporation in the Manufacturing industry. To address this disparity, we come to our next metric.\n\n---\n<strong id=\"#metric8\"><em>Metric 8: Corporate Electricity Greed</em></strong>  \n_Measure of corporate electricity consumption as a percentage of the global electricity poverty._\n<br><br>\n\n$\\begin{align} & CorporateElectricityGreed = \\frac{Total Electricity Consumption_{MWh}}{GlobalElectricityPoverty} \\times 100, \\\\ \\\\ \\text{where } \\\\ & GlobalElectricityPoverty = GlobalPerCapitaElectricityConsumption_{MWh} \\times PeopleWithoutElectricity \\end{align}$\n\n**Rationale**  \nLarge corporations are amongst the topmost consumers of electricity in the world. But a standard measure of the amount of electricity consumed doesn't give us any more information than the figures of consumption in say MWh. One useful way to hold companies accountable for their electricity use would be to measure their total consumption as a factor of the global electricity poverty which we define as - _the number of MWh of additional electricity needed to provide adequate access to people currently without electricity._ \n\n---\nAs such, _Corporate Electricity Greed_ directly draws attention to the divide between the consumerism driven mindsets of large corporations and the resource inadequacy and unavailability for the poorest sections of society. "},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# considering only corporates reporting at least some electricity consumption\ncc8_2a = cc8_2a[cc8_2a[total] > 0]\n\n# converting per capita consumption to MWh\nper_capita_MWh = global_pc_electricity_consumption['Per capita electricity (kWh)']/1000\n\n# calculating electricity greed by dividing corporate consumption by electricity poverty \n# electricity poverty = global per capita consumption X people without electricity\ncc8_2a['Greed'] = cc8_2a[total]*100/(per_capita_MWh * people_without_electricity)\n\ncc8_2a.reset_index().sort_values('Greed', ascending=False)[:20] \\\n.plot.barh(x='organization', y='Greed', figsize=(16,11), legend=False, color=palette[::2] + [grey]*19,\n          title='Top 20 Corporations by Corporate Electricity Greed', logx=True);\nplt.plot([100, 100], [0.29, 20], linewidth=5, c=palette[1])\nplt.annotate('Threshold for consumption exceeding Global Electricity Poverty', (106, 17), \n             rotation=-90, fontsize=13, fontweight=550)\nplt.gca().invert_yaxis()\n\n# cc8_2a[['C8.2a_C2MWh from renewable sources', 'C8.2a_C3MWh from non-renewable sources']] \\\n# .astype('float32').groupby(['row_name']).sum() \\\n# .plot(kind='barh', figsize=(8, 6), title='Global energy consumption by organizations by source (2020)');","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Visualizing the corporations ranked by electricity greed shows just how vast the divide is, with the top two corporates, Copper Tire & Rubber, and HudBay Minerals Inc., consuming roughly 2x and 0.5x the electricity required to provide enough to every single person who currently lacks it.\n\nIn addition to such indirect, comparitive, issues related to consumption, corporate energy activities have even more serious and direct impacts on human health, and that has to do with the fuels they use to fulfil their energy requirements. These will be addressed with the next metric.\n\n---\n<strong id=\"#metric9\"><em>Metric 9: Deaths by Corporate Fuel Use</em></strong>  \n_Number of deaths casused due to corporations' choice of fuels for energy generation._\n<br><br>\n\n$NoOfDeaths = \\sum_{i=1}^{fuels}Fuel Consumption(TWh)_{i} \\times DeathsPerTWh_{i}$\n<br><br>\n\n**Rationale**  \nMost of the world's energy continues to be generated by fossil fuels. As per [Markandya, A., & Wilkinson, P. (2007)](https://www.academia.edu/12681524/Electricity_generation_and_health), the use of fossil fuels for energy generation is directly linked to deaths in people due to air pollution related illnesses, or accidents in low visibility conditions. _Deaths by Corporate Fuel Use,_ thereby, provides a means for corporations to assess their impact on human health and safety by enabling them to calculate the number of deaths they might be responsible for due to their energy consumption habits. \n\n---\n\nThe following data on the fuel specific deaths per TWh of energy generated, is sourced from [Our World In Data](https://ourworldindata.org/grapher/death-rates-from-energy-production-per-twh), who themselves partially sourced it from the document linked above."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# avg. deaths per TWh of energy generation \n# https://ourworldindata.org/grapher/death-rates-from-energy-production-per-twh\nfuel_deaths = pd.DataFrame([\n    ('Biofuels', 0.0048),\n    ('Biomass', 4.63),\n    ('Brown coal', 32.72),\n    ('Coal', 24.62),\n    ('Gas', 2.821),\n    ('Hydropower', 0.0235),\n    ('Nuclear', 0.071),\n    ('Oil', 18.43),\n    ('Solar', 0.019),\n    ('Wind', 0.035)\n], columns=['Fuel Category', 'Deaths per TWh'])\n\nfuel_deaths.style.background_gradient(cmap='Reds')","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# mapping top fuels (by use) from CDP Questionnare Responses to the above given categories\nFUELS = pd.DataFrame([\n    ('Biogas', 'Biofuels'),\n    ('Biomethane', 'Biofuels'),\n    ('Bioethanol', 'Biofuels'),\n    ('Biodiesel', 'Biofuels'),\n    ('Biodiesel Tallow', 'Biofuels'),\n    ('Biodiesel Waste Cooking Oil', 'Biofuels'),\n    ('Bamboo', 'Biofuels'),\n    ('Black Liquor', 'Biofuels'),\n    ('Vegetable Oil', 'Biofuels'),\n    ('Animal Fat', 'Biofuels'),\n    ('Hardwood', 'Biofuels'),\n    ('Softwood', 'Biofuels'),\n    ('Wood', 'Biofuels'),\n    ('Wood Logs', 'Biofuels'),\n    ('Bagasse', 'Biofuels'),\n    ('Liquid Biofuel', 'Biofuels'),\n    ('Straw', 'Biofuels'),\n    ('Biogasoline', 'Biomass'),\n    ('Solid Biomass Waste', 'Biomass'),\n    ('Biomass Municipal Waste', 'Biomass'),\n    ('Dried Sewage Sludge', 'Biomass'),\n    ('Sludge Gas', 'Biomass'),\n    ('Wood Pellets', 'Biomass'),\n    ('Wood Chips', 'Biomass'),\n    ('Wood Waste', 'Biomass'),\n    ('Agricultural Waste', 'Biomass'),\n    ('General Municipal Waste', 'Biomass'),\n    ('Lignite Coal', 'Brown coal'),\n    ('Brown Coal Briquettes (BKB)', 'Brown Coal'),\n    ('Coal', 'Coal'),\n    ('GCI Coal', 'Coal'),\n    ('PCI Coal', 'Coal'),\n    ('Coke', 'Coal'),\n    ('Subbituminous Coal', 'Coal'),\n    ('Thermal Coal', 'Coal'),\n    ('Thermal Coal Commercial', 'Coal'),\n    ('Thermal Coal Domestic', 'Coal'),\n    ('Thermal Coal Industrial', 'Coal'),\n    ('Anthracite Coal', 'Coal'),\n    ('Bituminous Coal', 'Coal'),\n    ('Coking Coal', 'Coal'),\n    ('Metallurgical Coal', 'Coal'),\n    ('Natural Gas', 'Gas'),\n    ('Natural Gas Liquids (NGL)', 'Gas'),\n    ('Natural Gasoline', 'Gas'),\n    ('Liquefied Natural Gas (LNG)', 'Gas'),\n    ('Compressed Natural Gas (CNG)', 'Gas'),\n    ('Fuel Gas', 'Gas'),\n    ('Crude Oil', 'Oil'),\n    ('Crude Oil Extra Heavy', 'Oil'),\n    ('Crude Oil Heavy', 'Oil'),\n    ('Crude Oil Light', 'Oil'),\n    ('Marine Fuel Oil', 'Oil'),\n    ('Light Distillate', 'Oil'),\n    ('Refinery Oil', 'Oil'),\n    ('Petroleum Coke', 'Oil'),\n    ('Petrol', 'Oil'),\n    ('Diesel', 'Oil'),\n    ('Motor Gasoline', 'Oil'),\n    ('Aviation Gasoline', 'Oil'),\n    ('Jet Gasoline', 'Oil'),\n    ('Jet Kerosene', 'Oil'),\n    ('Residual Fuel Oil', 'Oil'),\n    ('Distillate Oil', 'Oil'),\n    ('Fuel Oil Number 1', 'Oil'),\n    ('Fuel Oil Number 2', 'Oil'),\n    ('Fuel Oil Number 4', 'Oil'),\n    ('Fuel Oil Number 5', 'Oil'),\n    ('Fuel Oil Number 6', 'Oil'),\n    ('Fuel Oil', 'Oil')\n], columns=['Fuel', 'Fuel Category'])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Using a mapping between the fuel types present in this data and the list of fuels provided in the CDP questionnaire, we can estimate the death rates for each fuel used by the corporation. Next, joining the mapping with corporate responses for fuels used in energy generation, lets us calculate the death tolls for each corporation."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# query corporate fuel use data\ncc8_2c = get_climate('C8.2c').pivot(index=['organization', 'row_number'], columns='column_name')['response_value']\n\n# convert from MWh to TWh\ncc8_2c['TWh of fuel consumed'] = (cc8_2c['C8.2c_C3Total fuel MWh consumed by the organization'].astype('float64'))/1000000\ncc8_2c = cc8_2c[cc8_2c['TWh of fuel consumed']!=0]\n\n# join (via fuel category mapping) with fuel deaths dataframe\ncc8_2c = cc8_2c[['C8.2c_C1Fuels (excluding feedstocks)', 'TWh of fuel consumed']] \\\n.join(FUELS.set_index('Fuel'), on='C8.2c_C1Fuels (excluding feedstocks)') \\\n.join(fuel_deaths.set_index('Fuel Category'), on='Fuel Category') \\\n.dropna(subset=['Fuel Category', 'TWh of fuel consumed'])\n\n# calculate number of deaths\ncc8_2c['Deaths'] = cc8_2c['TWh of fuel consumed'] * cc8_2c['Deaths per TWh']\n\ncc8_2c.groupby('organization')[['Deaths']].sum().round().astype('int32'). \\\nsort_values('Deaths', ascending=False)[:20].style.background_gradient(cmap='Reds') \\\n.set_caption('Number of (annual) deaths caused by corporate fuel use')\n\n# ccpiv8_2c.groupby(['organization', 'C8.2c_C1Fuels (excluding feedstocks)'])['C8.2c_C3Total fuel MWh consumed by the organization'].sum() \\\n# .sort_values(ascending=False)[:120].unstack() \\\n# .plot(kind='barh', stacked=True, figsize=(10,14), cmap='tab20', title='Breakdown of fuels consumption in MWh by organizations (2020)');","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Just like we saw in the Emissions section, the top culprits are corporations utilizing large quantities of fossil-fuels for energy generation and transportation.\n\n___\n\nWith nearly a billion people still without access to electricity in the developing world, the sheer scale of corporate energy consumption makes them come off as extremely complicit. Meanwhile, in energy poor areas, besides the obvious hardships, the use of dirty or polluting fuels has serious consequences on people’s health and wellbeing by exacerbating respiratory and cardiac illnesses, and mental health issues.\n\nFurther, by calculating _Deaths by Fuel Use,_ we discover the magnitude of corporate impact on people's lives due large scale corporate fossil-fuel consumption. The number of people affected by the activities of corporations such as Vistra Corp., Duke Energy and American Electric Power Company, shows the toxic nature of the corporations that value profits over life. \n\nIn view of all of this, we advocate that corporations need to take yet more notice of the extended consequences of their actions, and take steps to address the energy access problem. This is not only the right thing to do, but also opens doors to opportunities. For example, The World Bank’s International Finance Corporation (IFC) recently estimated the market size for modern energy services at US$37 billion – roughly equivalent to the amount those without access today spend on kerosene for lighting and biomass for cooking. In addition to the direct revenue opportunities providing access to energy bring, the domino effects - namely broader economic and socio-economic development - can lead to yet more opportunities.[<sup>[Link]</sup>](https://accntu.re/33rApI0)\n\n\n<h2 id=\"water-risks\">7.3 Water Risks</h2>\n\n<img src=\"https://i.pinimg.com/750x/0d/fd/51/0dfd512f7fb956eacbdf20afefe147ce.jpg\">\n\nIllustration by [Sebastien Thibault](https://www.behance.net/sebastienthibault)\n<br>\n\n<h3 id=\"water-costs-of-electricity\">7.3.1 Hidden Water Costs of Electricity</h3>\n\nThere have been numerous reports in the past year that talk about the depleting water sources, rising droughts and future water shortages to be faced by the world. However, overall action in safeguarding this dwindling resource is falling far short, and according to CDP,[<sup>[Link]</sup>](https://www.reutersevents.com/sustainability/global-water-crisis-looms-business-usual-no-longer-option) there has been a 50% rise in companies reporting higher water withdrawals since 2015.\n\nCorporations need to take more ambitious steps in order to counteract the impact they have on the planet's water resources, and it isn't just the social and environmental impacts of unsustainable water use that should drive companies to act. Water-related financial losses are rising. In 2018, companies reported such losses to the tune of $38 billion, and even after excluding the top 2 contributors, losses were still higher than in 2017.[<sup>[Link]</sup>](https://www.ipcc.ch/sr15/chapter/chapter-3/) Through this section, we aim to develop a metric that can be used as a mitigative tool for corporations to track the water consumed for electricity generation.\n\n---\n<strong id=\"#metric10\"><em>Metric 10: Water Cost of Electricity</em></strong>  \n_Gallons of upstream water consumed for purposes relating to the generation of electricity._\n<br><br>\n\n$$\\begin{align} & Water Cost_{gallons} = \\sum_{i=1}^{fuels}ElectricityConsumed(kWh)_{i} \\times WaterConsumptionFactor_{i}, \\\\ \n \\text{where, } \\\\\n&WaterConsumptionFactor_{i} = \\text{Gallons of water consumed per kWh electrcity from fuel }i \\end{align} $$\n<br><br>\n\n**Rationale**  \nFreshwater use for electricity generation accounts for a large proportion of total water consumption across the world. At the same time, the world is not on track to meet SDG 6 - securing water and sanitation for all by 2030, with 1 in 3 people globally without access to safe drinking water.[<sup>[Link]</sup>](https://www.unicef.org.uk/press-releases/1-in-3-people-globally-do-not-have-access-to-safe-drinking-water-unicef-who/#:~:text=It%20is%20estimated%20that%201,who%20drink%20untreated%20surface%20water) The _Water Cost of Electricity_ allows us to measure the hidden upstream water consumption related to energy generation and use by corporations which may otherwise be used to satisfy the basic water-related needs of people.\n    \n---\n\nData on the Water Consumption Factor for fuels is sourced from WRI. Their approach and calculations for the factor score are described in [Guidance for Calculating Water Use Embedded in Purchased Electricity](https://www.wri.org/publication/guidance-calculating-water-use-embedded-purchased-electricity). Since CDP has provided response data only for organizations in the USA and Canada, water consumption factor values for these 2 countries are all that we will need. These are displayed in the table given below."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"water_factor.unstack()['Water Consumption Factor'].drop(['Geothermal', 'Hydro', 'Nuclear', 'Photovoltaics', 'Solar Thermal', 'Wind'], axis=1) \\\n.style.background_gradient('OrRd', axis=1).set_caption('Water Consumption Factor for Fuels (gal/kWh)').set_precision(3)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"To use the water factor values, we must once again create a mapping between the dataset and the responses. Although all fuels from the CDP Questionnaire are not listed in the water factor dataset, we can make some assumptions in order to map them to the most similar fuels.\n\nFor example, Bituminous and Subbituminous coal have similar calorific content compared to Hard Coal (Anthracite), and hence, would result in a similar water consumption factor.\n\n<img src=\"https://upload.wikimedia.org/wikipedia/commons/thumb/2/21/Coal_Rank_USGS.png/1024px-Coal_Rank_USGS.png\" alt=\"Coal Caloric Value\" width=\"400\"/>\n\n<br><br>\n\nIn some of the other cases, we map high-grade derivatives of a fuel to the fuel itself (Natural Gas) with the assumption that derivatives would, in general, have a higher water factor than the raw fuel. The mapping now allows us to calculate our metric. Plotting the top 20 corporations by embedded water consumption shows that the group consists almost exclusively of energy generation companies, which makes a lot of sense."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# mapping fuels from CDP questionnare to fuels provided in water consumption factor dataset\nFUELS2 = pd.DataFrame([\n    ('Anthracite Coal', 'Hard Coal'),\n    ('Bituminous Coal', 'Hard Coal'),\n    ('Subbituminous Coal', 'Hard Coal'),\n    ('Coal', 'Hard Coal'),\n    ('Lignite Coal', 'Lignite'),\n    ('Peat', 'Peat'),\n    ('Natural Gas', 'Natural Gas'),\n    ('Fuel Gas', 'Natural Gas'),\n    ('Natural Gas Liquids (NGL)', 'Natural Gas'),\n    ('Natural Gasoline', 'Natural Gas'),\n    ('Liquefied Natural Gas (LNG)', 'Natural Gas'),\n    ('Compressed Natural Gas (CNG)', 'Natural Gas'),\n    ('Solid Biomass Waste', 'Biomass'),\n    ('Biomass Municipal Waste', 'Biomass'),\n    ('Dried Sewage Sludge', 'Biomass'),\n    ('Sludge Gas', 'Biomass'),\n    ('Wood Pellets', 'Biomass'),\n    ('Wood Chips', 'Biomass'),\n    ('Wood Waste', 'Biomass'),\n    ('Agricultural Waste', 'Biomass'),\n    ('Hardwood', 'Biomass'),\n    ('Softwood', 'Biomass'),\n    ('Wood', 'Biomass'),\n    ('Wood Logs', 'Biomass'),\n    ('Straw', 'Biomass'),\n    ('Wood', 'Biomass'),\n    ('Wood Waste', 'Biomass'),\n    ('Landfill Gas', 'Biogas'),\n    ('Biogas', 'Biogas'),\n    ('Biomethane', 'Biogas'),\n    ('General Municipal Waste', 'Waste'),\n    ('Waste Tires', 'Waste'),\n    ('Industrial Wastes', 'Waste'),\n    ('Non-Biomass Municipal Waste', 'Waste'),\n    ('Non-Biomass Waste', 'Waste'),\n    ('Waste Paper and Card', 'Waste'),\n    ('Waste Plastics', 'Waste'),\n    ('Residual Fuel Oil', 'Heavy Fuel Oil'),\n    ('Town Gas', 'Coal Gases')\n], columns=['Fuel', 'Fuel Type'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# query corporate fuel use data again\ncc8_2c = get_climate('C8.2c', cols=['country', 'organization']) \\\n.pivot(index=['country', 'organization', 'row_number'], columns='column_name')['response_value']\n\n# convert electricity consumption from MWh to KWh\ncc8_2c['kWh of electricity generation'] = cc8_2c['C8.2c_C4MWh fuel consumed for self-generation of electricity'].astype('float64') * 1000\ncc8_2c = cc8_2c[cc8_2c['kWh of electricity generation']!=0]\n\n# join with fuel type mapping above \ncc8_2c = cc8_2c[['C8.2c_C1Fuels (excluding feedstocks)', 'kWh of electricity generation']] \\\n.join(FUELS2.set_index('Fuel'), on='C8.2c_C1Fuels (excluding feedstocks)').dropna(subset=['Fuel Type', 'kWh of electricity generation']) \\\n.reset_index().set_index(['country', 'Fuel Type'])\n\n# add water factor data for each fuel\ncc8_2c['water_factor'] = water_factor['Water Consumption Factor']\n\n# calculate gallons consumed for electricity generation\ncc8_2c['gallons'] = cc8_2c['kWh of electricity generation'] * cc8_2c['water_factor']\n\ncc8_2c.groupby(['organization'])['gallons'].sum().sort_values(ascending=False)[:20] \\\n.plot.barh(color=palette[:2] + [grey]*18,\n           title='Gallons of Upstream Water Consumption linked to Electricity Generation');\nplt.gca().invert_yaxis()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"It must be noted that these results are somewhat incomplete. Even though we have tried our best to include a majority of the fuels from the CDP Questionnaire, all haven't been accounted for. Sadly, there was no way around this as the WRI data was the best we could find to calculate embedded water consumption in energy use. \n\nOn the plus side, despite of the gap in the data, the KPI does well to highlight just how integral water is to electricity generation. Moreover, it could be applied to calculate hidden water costs for organization facilities located near stressed water resources and generating their own electricity. We would have done this ourselves, using the responses to question `W5.1` of the Water Security Questionnaire, however, there wasn't any data on the total amount of electricity generated/consumed by the facilities. Maybe the 2021 Questionnaire could add this field! :)"},{"metadata":{},"cell_type":"markdown","source":"<h3 id=\"water-stress\">7.3.2 Water Stress</h3>\n\n<img src=\"https://files.wri.org/s3fs-public/styles/large/public/aqueduct_projections_20150409.png?itok=WISZGyCc\"> \n\nImage from [WRI](https://www.wri.org/)\n<br>\n\nWater is a ubiquitous resource. However, massive quantities of it are consumed by multinational corporates to sustain the global scale of their operations. It is, therefore, to be expected that some, if not all, of this water is sourced from water-stressed regions. These may be overstressed river basins, or countries facing issues related to water scarcity at the national level. Due to this, it becomes essential for corporations to account for the level of their water consumption from stressed regions or resources.\n\nWe can use the responses provided by organizations, to visualize their facilities by their total water consumption in stressed and non-stressed regions - as reported by them."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# using w, c, d for withdrawn, consumption, discharge respectively\nw = 'Total water withdrawals at this facility (megaliters/year)'\nc = 'Total water consumption at this facility (megaliters/year)'\nd = 'Total water discharges at this facility (megaliters/year)'\n\n# querying corporate responses for water accounting data\nw5_1 = get_water('W5.1').pivot(index=['organization', 'account_number', 'row_number'], columns='column_name')['response_value']\nw5_1 = w5_1[[\n    'W5.1_C3Country/Area & River basin_G', 'W5.1_C3Country/Area & River basin', \n    'W5.1_C6Located in area with water stress', 'W5.1_C4Latitude', 'W5.1_C5Longitude',\n    'W5.1_C9' + w, 'W5.1_C23' + c, 'W5.1_C17' + d\n]]\n\n# renaming columns to look nice on visualizations\nw5_1.columns= [\n    'Country/Area',\n    'River basin',\n    'Water Stressed Area',\n    'Latitude',\n    'Longitude',\n    w, c, d\n]\n\nw5_1['Water Stressed Area'] = w5_1['Water Stressed Area'].fillna('Unknown')\n\nfor col in w5_1.columns[3:]:\n    w5_1[col] = w5_1[col].astype('float32')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"w5_1c = w5_1[w5_1[c]>0]\n\n# size of plot in figure is a function of water consumption\nw5_1c['size'] = np.log10(1 + w5_1c[c])\n\n# visualizing facilites by their consumption and water stress\nfig =  px.scatter_geo(w5_1c, lat='Latitude',\n               lon='Longitude', \n               color='Water Stressed Area',\n               hover_data=[w, c, 'Country/Area', 'River basin'],\n#                projection=\"natural earth\",        \n               size='size',\n               size_max=12,\n               color_discrete_sequence=palette[:2] + [grey],\n               title='Self reported water consumption by facilities from stressed/non stressed water resources',\n)\nfig.update_layout(height=480, margin={\"r\":0,\"t\":30,\"l\":0,\"b\":0})\nfig.update_geos(resolution=50, showcountries=True, countrywidth=0.2, showrivers=True, showlakes=True, \n                showframe=False, showcoastlines=False, rivercolor='#97B9DD')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Given that a majority of these facilities are located in water stressed regions, we present a way for corporations to evaluate their water consumption.\n\n---\n<strong id=\"#metric11\"><em>Metric 11: Unsustainable Water Use</em></strong>  \n_Measure of water consumption from water-stressed regions and resources._\n<br><br>\n\n$$UnsustainableWaterUse_{riverBasins} = \\sum_{i=1}^{facilites}(WaterStressScore_{riverBasin_{i}} \\times \\log_{10}(1 + WaterConsumption_{i}))$$\n<br><br>\n$$UnsustainableWaterUse_{countries} = \\sum_{i=1}^{facilites}(WaterStressScore_{country_{i}} \\times \\log_{10}(1 + WaterConsumption_{i}))$$\n<br><br>\n\n**Rationale**\n_Unsustainable Water Use_ takes into consideration the water stress at the river basin and country level for each facility belonging to a corporation, and assigns a pair of scores to the company tied to their total consumption from all facilities. \n\n---\n\nThe water stress scores for the largest, most populous river basins, and for all of the countries are available on WRI as part of their [Aqueduct Country and River Basin Rankings](https://www.wri.org/resources/data-sets/aqueduct-country-and-river-basin-rankings). We use this data to first visualize water stress at the country level."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# load water stress data for river basins, countries\nwater_stress_basin = pd.read_csv('../input/aqueduct-baseline-water-stress-20132014/aqueduct_basins_baseline_water_stress_20131024.csv').set_index('Name')\n\nwater_stress_country = pd.read_csv('../input/aqueduct-baseline-water-stress-20132014/aqueduct_countries_baseline_water_stress_20140109.csv') \\\n.set_index('Name').replace('-', np.nan).astype('float32')\n# add iso codes to country water stress data\nwater_stress_country = water_stress_country.join(iso_country, on='Name')[['All sectors', 'alpha-3']]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# visualize baseline water stress country level data\nfig_c = px.choropleth(water_stress_country.reset_index(), locations=\"alpha-3\",\n        color=\"All sectors\", # lifeExp is a column of gapminder\n        hover_name=\"Name\", # column to add to hover information\n#         projection=\"natural earth\",\n#         projection='orthographic',\n        color_continuous_scale=px.colors.sequential.Reds,            \n        title='Baseline water stress at country level',\n)\nfig_c.update_layout(height=480, margin={\"r\":0,\"t\":30,\"l\":0,\"b\":0})\nfig_c.update_geos(resolution=50, showcountries=True, countrywidth=0.2, showrivers=True, showlakes=True,\n                showframe=False, showcoastlines=False,  rivercolor='#97B9DD')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"It can be seen that large swathes of the world are facing high levels of water stress, and barring Central Africa and parts of South America, all continents have above-median baseline water stress scores. These scores will be used as weights for calculating each facilities' contribution towards the corporation's _Unsustainable Water Consumption,_ but to use the water stress data, we must, once again, create a mapping between the river basins in the dataset and those provided in the CDP Questionnaire. The mapping will allow us to directly associate facilities to their corresponding river basin's baseline stress scores as provided in the Aqueduct data."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# mapping between river basins in CDP Questionnaire and Water Stress data\nRIVER_BASINS = pd.DataFrame([\n    ('Mississippi River', 'Mississippi River'),\n    ('St. Lawrence', 'St.Lawrence'),\n    ('Yangtze River (Chang Jiang)', 'Yangtze River (Chang Jiang)'),\n    ('Sacramento River - San Joaquin River', 'Sacramento San Joaquin'),\n    ('Colorado River (Pacific Ocean)', 'Colorado River (Pacific Ocean)'),\n    ('Krishna', 'Krishna'),\n    ('Ganges - Brahmaputra', 'Ganges Brahmaputra'),\n    ('Santiago', 'Santiago'),\n    ('Indus', 'Indus'),\n    ('Bravo', 'Bravo'),\n    ('Mackenzie River', 'Mackenzie River'),\n    ('Brazos River', 'Brazos River'),\n    ('Godavari', 'Godavari'),\n    ('Nile', 'Nile'),\n    ('Yongding He', 'Yongding He'),\n    ('Huang He (Yellow River)', 'Huang He (Yellow River)'),\n    ('Han-Gang (Han River)', 'Han-Gang (Han River)'),\n    ('Columbia River', 'Columbia River'),\n    ('Parana', 'Parana'),\n    ('Murray - Darling', 'Murray'),\n    ('Other, please specify: Sabarmati', 'Sabarmati River'),\n    ('Wisla', 'Wisla'),\n    ('Colorado River (Pacific Ocean)', 'Colorado River (Pacific Ocean)'),\n    ('Liao He', 'Liao He'),\n    ('Nelson River', 'Nelson River'),\n    ('Delaware River', 'Delaware River'),\n    ('Other, please specify: Rio Grande - Bravo', 'Bravo'),\n    ('Other, please specify: Santiago Guadalajara', 'Santiago'),\n    ('Other, please specify: Huangpu River', 'Huangpu Jiang'),\n    ('Dead Sea', 'Dead Sea (Jordan)'),\n    ('Brantas', 'Brantas'),\n    ('Other, please specify: Sabarmati - Sabarmati', 'Sabarmati River'),\n    ('Other, please specify: Pennar (Cauvery To Krishna)', 'Krishna'),\n    ('Other, please specify: Sacramento', 'Sacramento San Joaquin'),\n    ('Other, please specify: Krishna - Delta', 'Krishna'),\n    ('Other, please specify: Several locations and basins: Krishna, South coast and East coast', 'Krishna'),\n    ('Other, please specify: Sabarmati River', 'Sabarmati River'),\n    ('Other, please specify: Krishna â€“ Musi/Aler', 'Krishna'),\n    ('Other, please specify: Krishna â€“ Ghod/ Nira/ Bhima/ Man', 'Krishna'),\n    ('Other, please specify: Krishna  - Krishna/Malpharba/Varna', 'Krishna'),\n    ('Other, please specify: Ganges Brahmaputra - Jamni', 'Ganges Brahmaputra'),\n    ('Other, please specify: AMA, Colorado River and San Juan Basin', 'Colorado River (Pacific Ocean)'),\n    ('Other, please specify: Mackenzie River Basin', 'Mackenzie River'),\n    ('Other, please specify: Multiple river basins including Yangtze', 'Yangtze River (Chang Jiang)'),\n    ('Other, please specify: Pearl and Yangtze', 'Yangtze River (Chang Jiang)'),\n    ('Other, please specify: Several locations and basins: Coast, Xun Jiang, Bo Hai, Ziya He, Huang He, Amur', 'Huang He (Yellow River)'),\n    ('Other, please specify: Multiple including Colorado and Mojave', 'Colorado River (Pacific Ocean)'),\n    ('Other, please specify: St. Lawrence', 'St.Lawrence'),\n    ('Other, please specify: AMAs, Colorado River, and San Juan River Basin', 'Colorado River (Pacific Ocean)'),\n    ('Other, please specify: AMAs, Colorado River and Jan Juan River Basin', 'Colorado River (Pacific Ocean)'),\n    ('Other, please specify: Citarum River Basin & Brantas River Basin', 'Brantas'),\n    ('Other, please specify: GHAASBasin 3724, 3725, 3736; Colorado', 'Colorado River (Pacific Ocean)'),\n    ('Other, please specify: Ganges Brahmaputra â€“ Kali Gandaki/ Seti', 'Ganges Brahmaputra'),\n    ('Other, please specify: Ganges Brahmaputra â€“ Lakhanadi/Ghuari', 'Ganges Brahmaputra'),\n    ('Other, please specify: Bravo, North Gulf, and Interior Basins', 'Bravo'),\n    ('Other, please specify: Colorado River', 'Colorado River (Pacific Ocean)')\n], columns=['river_basin_cdp', 'river_basin'])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Using the mapping, we are able to calculate the _Unsustainable Water Use_ at the basin level, and see that Coca-Cola ranks first by cumulative score across all river basins and facilities. (Probably because of all that coke we've been drinking while we worked...)"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# joining with water stress data for river basins via river basin mapping\n# ignoring facilities with no consumption\nw5_1a = w5_1[w5_1[c] > 0].join(RIVER_BASINS.set_index('river_basin_cdp'), on='River basin') \\\n.join(water_stress_basin['All sectors'], on='river_basin')\n\n# fill values for river basins not present in water stress data with upper & lower quantile scores for Yes and No, and mean score for Unknown\nw5_1a['All sectors'].fillna(w5_1a['Water Stressed Area'].map({\n    'Yes': water_stress_basin['All sectors'].quantile(0.75), \n    'Unknown': water_stress_basin['All sectors'].mean(), \n    'No': water_stress_basin['All sectors'].quantile(0.25)\n}), inplace=True)\n\n# calculating unsustainable water consumption score for each facility\nw5_1a['score'] = w5_1a['All sectors'] * np.log10(1 + w5_1a[c]) #* w5_1a[c]/w5_1a[w]\n# w5_1a.sort_values(c, ascending=False)\nw5_1a.groupby('organization')['score'].sum().round(3).sort_values(ascending=False)[:20] \\\n.plot.barh(color=palette[:1]+[grey]*19,\n          title='Top 20 companies by Unsustainable Water Use (River Basin)');\nplt.gca().invert_yaxis()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The facilities can once more be mapped out, but this time coloured by how they score on _Unsustainable Water Use,_ and overlaid on the baseline water stress at the country level. This enhances our understanding regarding the _contribution_ of these facilities towards increasing the water stress in already stressed regions."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# calculating average scores for each river basin\nw5_1plot = w5_1a.groupby(['Country/Area', 'River basin', 'Water Stressed Area',\n               'Latitude', 'Longitude'])[[c, 'score']].mean().reset_index()\n\n# visualizing unsustainable water consumption\nfig = px.scatter_geo(w5_1plot, lat='Latitude',\n               lon='Longitude', \n               color='score',\n               hover_data=[c, 'Country/Area', 'River basin'],\n#                projection=\"natural earth\",        \n               size='score',\n               size_max=14,\n               title='Average Unsustainable Water Use scores for facilities',\n               color_continuous_scale=px.colors.sequential.Reds,\n\n)\nfig.update_layout(height=480, margin={\"r\":0,\"t\":30,\"l\":0,\"b\":0})\nfig.update_geos(resolution=50, showcountries=True, countrywidth=0.2, showrivers=True, showlakes=True,\n                showframe=False, showcoastlines=False,  rivercolor='#97B9DD')\n\nfig.add_trace(\n    fig_c.data[0]\n)\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Similarly, we can find country-level _Unsustainable Water Use_ by scoring corporation water use by the country/region they operate in. The scores for the top 20 companies for any single region are presented below. "},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# change country names to match  water stress data\nw5_1['Country/Area'] = w5_1['Country/Area'].replace({\n    'Republic of Korea': 'South Korea',\n    'Taiwan, Greater China': 'Taiwan',\n    'Puerto Rico': '',\n    'United Kingdom of Great Britain and Northern Ireland': 'United Kingdom',\n    'Russian Federation': 'Russia',\n    'CÃ´te d\\'Ivoire': 'Ivory Coast',\n    'Viet Nam': 'Vietnam',\n    'North Macedonia': 'Macedonia',\n    'China, Hong Kong Special Administrative Region': 'Hong Kong'\n})","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# join water consumption data with water stress scores and iso codes\n# ignoring facilities with no consumption\nw5_1b = w5_1[w5_1[c] > 0].join(water_stress_country[['All sectors', 'alpha-3']], on='Country/Area')\n\n# fill values for countries/regions not present in water stress data with quantile scores for Yes and No, and mean score for Unknown\nw5_1b['All sectors'].fillna(w5_1b['Water Stressed Area'].map({\n    'Yes': water_stress_country['All sectors'].quantile(0.75), \n    'Unknown': water_stress_country['All sectors'].mean(), \n    'No': water_stress_country['All sectors'].quantile(0.25)\n}), inplace=True)\n\n# calculate unsustainbable water use score\nw5_1b['score'] = w5_1b['All sectors'] * np.log10(1 + w5_1b[c]) # * w5_1b[c]/w5_1b[w]\n# create counter to keep track of number of facilities\nw5_1b['facilities'] = 1\n\nw5_1b.groupby(['organization', 'Country/Area'])[['score', c, w, d, 'facilities']].sum() \\\n.sort_values('score', ascending=False)[:20].unstack().plot.barh(y='score', stacked=True, figsize=(12, 9), \ncolor=[qualitative[-3:-2]]+ palette, title='Top 20 companies with highest Unsustainable Water Use in a single Country').legend(title='Country')\nplt.gca().invert_yaxis()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The data skews highly towards the USA simply because the responses are from companies located in the USA and Canada, and it is likely that most of the facilties will be located there too. However, we can still calculate the _mean Unsustainable Water Use_ score for each country to get an idea about how much an average facility located there might be impacting water resources. "},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# calculate average score for each country\nw5_1plot2 = w5_1b.groupby(['Country/Area', 'alpha-3'])[['score', c, 'facilities']].sum().reset_index()\nw5_1plot2['mean_score'] = w5_1plot2['score']/w5_1plot2['facilities']\n\n# visualize average country level unsustainable water consumption\nfig = px.choropleth(w5_1plot2, locations=\"alpha-3\",\n                    color=\"mean_score\",\n                    hover_data=[\"Country/Area\", 'facilities'],\n                    color_continuous_scale=px.colors.sequential.OrRd,\n#                     projection='natural earth',\n                    title='Unsustainable water consumption at country level'\n                   )\nfig.update_layout(height=440, margin={\"r\":0,\"t\":30,\"l\":0,\"b\":0})\nfig.update_geos(resolution=50, showcountries=True, countrywidth=0.2, showrivers=True, showlakes=True,\n                showframe=False, showcoastlines=False, rivercolor='#97B9DD')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"---\n\nClean freshwater is essential for a healthy human life, however, 1.1 billion people lack access to water and 2.7 billion experience water scarcity at least one month a year.[<sup>[Link]</sup>](https://wwf.to/36vR3YW) The analysis conducted in this section shows how large corporations use massive quantities of water for electricity generation and to conduct their operations. In many of the cases, this water comes from water-stressed areaa, resulting in further aggravation of water-scarcity related issues in these basins, and affecting the local populations and the ecology dependent on these water sources. \n\n---\n\n<h1 style=\"color: #FFFFFF\" id=\"collaboration\">8 Collaboration</h1>\n\n<img src=\"https://media.istockphoto.com/vectors/raised-hands-helping-hands-team-work-collaboration-illustration-vector-id830356322?k=6&m=830356322&s=170667a&w=0&h=Co7OUUpoEjo0Q3ZGPA7liB7e8XIOX3slavvNEycBnzg=\">\n\n\nIllustration from [iStock](https://media.istockphoto.com/vectors/raised-hands-helping-hands-team-work-collaboration-illustration-vector-id830356322)\n<br>\n\nLooking at the previous sections we see a very disturbing trend of corporate overconsumption against a backdrop of inequality. In all these cases, corporate actions such as GHG emissions, use of unclean fuels, and over consumption of water from stressed resources, are seen to disproportionately affect those who are already prone to worst affects of climate change. This makes it clear that climate change, in addition to being a health issue and an existential issue, is more importantly a social issue. It is a challenge that is extremely complex and does not have a one size fits all solution, but it's a challenge that can drive human innovation to create sustainable solutions.\n\nUsing the Questionnaire data, we start by looking at the ways cities are already collaborating with businesses in the city and then create metrics that encourage collaborative actions."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# query city respinse data on collaborations\nc6_2a = get_cities('6.2a').pivot(index=['City', 'Account Number', 'Row Number'], columns=['Column Name'])['Response Answer']\n\n# replaceing all self defined catergories with 'Others'\nc6_2a.loc[c6_2a['Type of collaboration'].str.contains('Other, ', na=False), 'Type of collaboration'] = 'Others'\n\ncollab_area = c6_2a.groupby('Collaboration area')['Type of collaboration'].value_counts().unstack()\ncollab_area.fillna(0).astype('int32').style.background_gradient('Reds') #.set_properties(**{'font-size': '20px'}) \n# sns.jointplot();\n\n# c6_2a['Type of collaboration'].value_counts()\n\n# plt.subplots(figsize=(12, 8))\n# sns.heatmap(c6_2a[cols], annot=True);\n# plt.xticks(rotation=65, horizontalalignment='right',)\n# plt.title('City and business collaborations (2020)')\n# plt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We find that several initiatives are focussed towards Energy. In addition, Transport, Waste, and Water, seem to be noteworthy areas for collaboration. We can see that cities are actively making efforts to collaborate with businesses, so it would be extremely useful if there was a way to measure collaboration ability between a city and a corporation. And so we present our final metric.\n\n---\n<strong id=\"#metric12\"><em>Metric 12: Collaboration Strength</em></strong>  \n_A measure of the joint interest and expected mutual benefits from collaborative partnerships between cities and corporations on shared risks, opportunities, and initiatives._\n<br><br>\n\n$$ CollaborationStrength_{city,org}  = SharedRisk \\times CommonOpportunities \\times \\mathtt{SERO}Score_{city,org},$$\n<br><br>\n$\\begin{align} &\\text{where } \\quad  \\mathtt{SERO}Score_{city,org}  = 5 -\\frac{\\sum_{i=1}^{mitigationActions_{city}}\\sum_{j=1}^{reductionInitiatives_{org}}\\mathtt{WordMoversDistance}(i, j)}{No.ofMitigationActions \\times No.ofReductionInitiatives} \\\\ \\\\ & \\text{and } \\quad org \\text{ is a corporation headquartered in } city \\\\ \\\\\n(\\mathtt{SERO} &= \\text{Similarity in Emission Reduction Objectives})\n\\end{align}$\n<br><br>\n\n**Rationale**  \n_Collaboration Strength_ is a metric that applies to pairs of cities and organizations, and thereby seeks to provide a score for the shared incentives that may be realized through joint initiatives. It is measured by taking into account the shared risks and objectives of cities with the corporations that have a presence in them. The shared objectives themselves are derived by finding overlaps between the city and corporate responses in the CDP Questionnaires. These include common opportunities and similarities in their strategies for emission reduction. _Collaboration Strength,_ therefore, enables cities to find corporations that are most suitable to work with to achieve their climate goals.\n\n---\n\nSince we only consider collaboration possibilities between corporations and cities which contain their headquarters, larger cities with high numbers of listed corporations are going to have more possibilites of finding suitable collaboration partners. The following visualization shows cities by the number of disclosing corporations headquartered in them."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"collab_frame = corp_locations.join(city_locations[['lon', 'lat']], on=['cleaned_address_city', 'hq_country'], how='inner')\n\nplot3 = collab_frame.groupby(['hq_country', 'cleaned_address_city', 'lon', 'lat'])['cleaned_name'].count().rename('HQ Count')\n\nfig =  px.scatter_geo(plot3.reset_index(), lat='lat',\n            lon='lon', \n            text='cleaned_address_city',\n            scope='north america',\n            size='HQ Count',\n            color_discrete_sequence=palette[:1],\n            title='Cities by number of reporting corporate HQs',\n)\nfig.update_layout(height=560, margin={\"r\":0,\"t\":30,\"l\":0,\"b\":0})\nfig.update_geos(showcountries=True, countrywidth=0.2, showcoastlines=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Collaboration by Shared Risks\n\nTo find the shared risks between cities and corporations, we return to the risk mapping we had used earlier in our vulnerability assessment for corporations. As a reminder, the mapping is between the climate change and water-related **physical risks** that corporations face and climate hazards and water security risks which cities are prone to. This can be used with the idea that corporations located in cities facing similar issues as them have a greater incentive to work together in tackling these problems."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# get corporate climate risk data\ncorp_climate_risks = get_climate('C2.3a').pivot(index=['organization', 'account_number', 'row_number'], columns='column_name') \\\n['response_value']['C2.3a_C3Risk type & Primary climate-related risk driver']\n\n# query data from questions on water risks (direct and indirect operations)\nw4_2 = get_water('W4.2').pivot(index=['organization', 'account_number', 'row_number'], columns='column_name')['response_value']\nw4_2a = get_water('W4.2a').pivot(index=['organization', 'account_number','row_number'], columns='column_name')['response_value']\n\n# standardizing columns\n\nw4_2.columns = ['Potential financial impact figure - minimum (currency)',\n       'Potential financial impact figure - maximum (currency)',\n       'Explanation of financial impact',\n       'Primary response to risk', 'Description of response',\n       'Cost of response', 'Explanation of cost of response',\n       'Country/Area & River basin',\n       'Country/Area & River basin_G',\n       'Type of risk & Primary risk driver',\n       'Type of risk & Primary risk driver_G',\n       'Primary potential impact',\n       'Company-specific description', 'Timeframe',\n       'Magnitude of potential impact', 'Likelihood',\n       'Are you able to provide a potential financial impact figure?',\n       'Potential financial impact figure (currency)']\n\nw4_2a.columns = ['Potential financial impact figure (currency)',\n       'Potential financial impact figure - minimum (currency)',\n       'Potential financial impact figure - maximum (currency)',\n       'Explanation of financial impact',\n       'Primary response to risk',\n       'Primary response to risk_G',\n       'Description of response', 'Cost of response',\n       'Explanation of cost of response',\n       'Country/Area & River basin',\n       'Country/Area & River basin_G', 'Stage of value chain',\n       'Type of risk & Primary risk driver',\n       'Type of risk & Primary risk driver_G',\n       'Primary potential impact',\n       'Company-specific description', 'Timeframe',\n       'Magnitude of potential impact', 'Likelihood',\n       'Are you able to provide a potential financial impact figure?']\n\n# combine the 2 dataframes\nw4_2concat = pd.concat([w4_2, w4_2a]).sort_values('organization')\n\n# get corporate water risk data\ncorp_water_risks = w4_2concat['Type of risk & Primary risk driver']\n\n# combine climate and water risks\ncorp_risks = pd.concat([corp_climate_risks, corp_water_risks]).dropna()\ncorp_risks = corp_risks.to_frame().join(corp_locations).reset_index().rename(columns={0: 'corporate_risk'})","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# get city climate hazards data\nc2_1 = get_cities('2.1', cols=['Country', 'City', 'Account Number'])\ncity_climate_hazards = c2_1[c2_1['Column Name']=='Climate Hazards'].set_index(['Country', 'City', 'Account Number'])['Response Answer']\n\n# get city water risk data\ncity_water_risks = get_cities('14.2a', cols=['Country', 'City', 'Account Number']) \\\n    .pivot(index=['Country', 'City', 'Account Number', 'Row Number'], columns='Column Name')['Response Answer']['Water security risk drivers']\n\n# concatenate climate and water risks, and remove data where no city name is given \ncity_risks = pd.concat([city_climate_hazards, city_water_risks]).to_frame().sort_index().reset_index().dropna().rename(columns={0: 'city_risk'})","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"By filtering for common location, and applying the risk mapping to match corporates with cities, we can measure the number of shared risks."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# join city data with corporate data via the risk mapping defined above\ncollab_frame['Shared Risks'] = corp_risks.join(risk_mapping.set_index('corporate_risk'), on='corporate_risk',  how='inner') \\\n         .join(city_risks.set_index(['city_risk', 'Country', 'City']), on=['city_risk', 'hq_country', 'cleaned_address_city'], how='inner') \\\n         .groupby(['Account Number', 'hq_country', 'cleaned_address_city', 'organization', 'account_number'])['corporate_risk'].count() \\\n         .reset_index([0,1,2], drop=True)\n\ncollab_frame.sort_values('Shared Risks', inplace=True)\ntemp = collab_frame.dropna(subset=['Shared Risks'])[-20:].reset_index()\n\nfig, ax = plt.subplots()\nax2 = ax.twinx()\ntemp.plot.barh(x='organization', y='Shared Risks', ax=ax, legend=False, xlabel='Organization',\n               title='Top Corporations and Cities ranked by number of Shared Risks');\n\ntemp.plot.barh(x='cleaned_address_city', y='Shared Risks', ax=ax2, legend=False,  \n               color=[grey]*18 + palette[:2], xlabel='City');","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Collaboration by Identified Opportunities\n\nLikewise, we create a mapping between the common opportunity drivers for cities and corporations. Similar opportunities which are likely to benefit both the city and the corporation would further their interests in collaborating. In the next visualization, we see the top cities and corporations that have similar or related opportunities"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# mapping between city and corporate opportunity drivers\nopportunity_mapping = pd.DataFrame([\n    ('Development of clean energy technologies/businesses', 'Use of more efficient modes of transport'),\n    ('Development of sustainable transport sector', 'Use of more efficient modes of transport'),\n    ('Development of circular economy models and businesses', 'Use of more efficient production and distribution processes'),\n    ('Development of local sustainable food businesses', 'Use of more efficient production and distribution processes'),\n    ('Increased opportunities for investment in infrastructure projects', 'Use of more efficient production and distribution processes'),\n    ('Increased opportunities for trade (nationally or internationally)', 'Use of more efficient production and distribution processes'),\n    ('Development of waste management sector', 'Use of recycling'),\n    ('Development of circular economy models and businesses', 'Use of recycling'),\n    ('Development of resource conservation and management', 'Use of recycling'),\n    ('Reduced risk to natural capital', 'Use of recycling'),\n    ('Development of energy efficiency measures and technologies', 'Move to more efficient buildings'),\n    ('Increased opportunities for investment in infrastructure projects', 'Move to more efficient buildings'),\n    ('Development of sustainable construction/real estate sector', 'Move to more efficient buildings'),\n    ('Development of water management sector', 'Reduced water usage and consumption'),\n    ('Increased water security', 'Reduced water usage and consumption'),\n    ('Development of resource conservation and management', 'Reduced water usage and consumption'),\n    ('Development of clean energy technologies/businesses', 'Use of lower-emission sources of energy'),\n    ('Development of energy efficiency measures and technologies', 'Use of lower-emission sources of energy'),\n    ('Creation/development of carbon markets', 'Use of supportive policy incentives'),\n    ('Development of clean energy technologies/businesses', 'Use of supportive policy incentives'),\n    ('Development of energy efficiency measures and technologies', 'Use of supportive policy incentives'),\n    ('Development of clean energy technologies/businesses', 'Use of new technologies'),\n    ('Development of energy efficiency measures and technologies', 'Use of new technologies'),\n    ('Carbon tax revenue', 'Participation in carbon market'),\n    ('Creation/development of carbon markets', 'Participation in carbon market'),\n    ('Development of energy efficiency measures and technologies', 'Shift toward decentralized energy generation'),\n    ('Increased energy security', 'Shift toward decentralized energy generation'),\n    ('Development of resource conservation and management', 'Development and/or expansion of low emission goods and services'),\n    ('Development of circular economy models and businesses', 'Development and/or expansion of low emission goods and services'),\n    ('Development of climate change resiliency projects', 'Development of climate adaptation, resilience and insurance risk solutions'),\n    ('Development of resource conservation and management', 'Development of climate adaptation, resilience and insurance risk solutions'),\n    ('Development of circular economy models and businesses', 'Development of new products or services through R&D and innovation'),\n    ('Increase opportunities for partnerships', 'Ability to diversify business activities'),\n    ('Increase opportunities for partnerships', 'Shift in consumer preferences'),\n    ('Increase opportunities for partnerships', 'Reputational benefits resulting in increased demand for goods/services [Financial services only]'),\n    ('Development of energy efficiency measures and technologies', 'Participation in renewable energy programs and adoption of energy-efficiency measures'),\n    ('Development of clean energy technologies/businesses', 'Participation in renewable energy programs and adoption of energy-efficiency measures'),\n    ('Development of resource conservation and management', 'Resource substitutes/diversification'),\n    ('Development of circular economy models and businesses', 'Resource substitutes/diversification'),\n    ('Development of climate change resiliency projects', 'Increased reliability, climate- resilience of investment chain'),\n    ('Development of water management sector', 'Cost savings'),\n    ('Increased opportunities for investment in infrastructure projects', 'Cost savings'),\n    ('Increased opportunities for investment in infrastructure projects', 'Improved field recovery factor (oil & gas sector only)'),\n    ('Increase opportunities for partnerships', 'Improved field recovery factor (oil & gas sector only)'),\n    ('Development of water management sector', 'Improved water efficiency in operations'),\n    ('Development of resource conservation and management', 'Improved water efficiency in operations'),\n    ('Increased water security', 'Improved water efficiency in operations'),\n    ('Development of water management sector', 'Water recovery from sewage management'),\n    ('Development of waste management sector', 'Water recovery from sewage management'),\n    ('Increased opportunities for investment in infrastructure projects', 'Water recovery from sewage management'),\n    ('Development of resource conservation and management', 'Water recovery from sewage management'),\n    ('Increased opportunities for investment in infrastructure projects', 'Increased resilience to impacts of climate change'),\n    ('Improved flood risk mitigation', 'Increased resilience to impacts of climate change'),\n    ('Development of climate change resiliency projects', 'Increased resilience to impacts of climate change'),\n    ('Development of climate change resiliency projects', 'Increased supply chain resilience'),\n    ('Additional funding opportunities', 'New R&D opportunities'),\n    ('Development of circular economy models and businesses', 'Reduced impact of product use on water resources'),\n    ('Increased water security', 'Reduced impact of product use on water resources'),\n    ('Reduced risk to natural capital', 'Reduced impact of product use on water resources')\n], columns=['city_opportunity', 'corp_opportunity'])\n\nopportunity_mapping[:20].style.set_caption('Sample from the opportunity mapping')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# creating city opportunity driver dataframe\ncity_opp = c6['Opportunity'].reset_index().set_index(['Opportunity', 'Country', 'City']).drop('Row Number', axis=1)\n\n# get climate chage related opportunity data \nc2_4a = get_climate('C2.4a').pivot(index=['organization', 'account_number', 'row_number'], columns='column_name')['response_value']\n\n# get water related opportunity data\nw4_3a = get_water('W4.3a').pivot(index=['organization', 'account_number', 'row_number'], columns='column_name')['response_value']\n\n# combining climate and water related opportunity drivers for corporates and joining with corporate location data\ncorp_opp = pd.concat([c2_4a['C2.4a_C4Primary climate-related opportunity driver'], w4_3a['W4.3a_C2Primary water-related opportunity']]) \\\n.reset_index().join(corp_locations, on=['organization', 'account_number']).drop('row_number', axis=1).rename(columns={0: 'corp_opp_driver'})\n\n# finding overlap between opportunity drivers for cities and corporates with HQs within them\ncollab_frame['Opportunities'] = corp_opp.join(opportunity_mapping.set_index('corp_opportunity'), on='corp_opp_driver', how='inner') \\\n        .join(city_opp, on=['city_opportunity', 'hq_country', 'cleaned_address_city'], how='inner') \\\n        .groupby(['Account Number', 'hq_country', 'cleaned_address_city', 'organization', 'account_number'])['corp_opp_driver'].count() \\\n        .reset_index([0,1,2], drop=True)\n\ncollab_frame.sort_values('Opportunities', inplace=True)\ntemp = collab_frame.dropna(subset=['Opportunities'])[-20:].reset_index()\n\nfig, ax = plt.subplots()\nax2 = ax.twinx()\n\ntemp[:20].reset_index().plot.barh(x='organization', y='Opportunities', ax=ax, legend=False, xlabel='Organization');\n\ntemp[:20].reset_index().plot.barh(x='cleaned_address_city', y='Opportunities', ax=ax2, legend=False,  color=[grey]*18 + palette[:1]*2,\n               title='City-Corporate pairs ranked by number of common opportunities');","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Collaboration by Emission Reduction Initiatives\n\nNext, we try to judge the similarity between city and corporate emission reduction strategies. For this, once we have grouped them by location, we calculate Word Mover's Distances (WMD) between each set of responses by a given city/corporate pair. Word Mover's Distance uses pre-trained Word Embeddings to calculate a distance (or negative similarity) score between strings, where an exact match returns 0. For our model, we use Google's [pre-trained word embeddings](https://code.google.com/archive/p/word2vec/). The average WMD distance between a city-corporate pair is then subtracted from 5 to invert values such that a higher similarity results in a hgher SERO score."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# query city data on emission reduction actions\nc5_4 = get_cities('5.4', cols=['Country', 'City', 'Account Number']) \\\n.pivot_table(index=['Country', 'City', 'Account Number', 'Row Number'], columns=['Column Name'],\n             aggfunc=lambda x: x.values[0] if len(list(x))==1 else list(x))['Response Answer']\n\n# create a dataframe by joining corporate initiatives with cities they're headquartered in\nemissions = piv4_3b[['C4.3b_C9Comment']].join(corp_locations) \\\n.join(c5_4.reset_index().set_index(['Country', 'City'])['Scope and impact of action'], on=['hq_country', 'cleaned_address_city'], how='inner')\n\n\n# remove special chars/punctuation/diacritics, numbers, spaces, convert to lowercase, remove stopwords\nemissions['city_initiative'] = emissions['Scope and impact of action'].fillna('') \\\n.apply(lambda x: re.sub(r'[!-/:-@\\[-`\\{-~‘’–]', '', x)).apply(lambda x: re.sub(r'[0-9]', '', x)) \\\n.apply(lambda x: re.sub(r'\\s+', ' ', x.lower().strip())).apply(lambda x: [y for y in x.split() if y not in stop])\n\nemissions['corp_initiative'] = emissions['C4.3b_C9Comment'].fillna('') \\\n.apply(lambda x: re.sub(r'[!-/:-@\\[-`\\{-~‘’–]', '', x)).apply(lambda x: re.sub(r'[0-9]', '', x)) \\\n.apply(lambda x: re.sub(r'\\s+', ' ', x.lower().strip())).apply(lambda x: [y for y in x.split() if y not in stop])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"# load the Word2Vec model\nmodel = gensim.models.KeyedVectors.load_word2vec_format('../input/googles-trained-word2vec-model-in-python/GoogleNews-vectors-negative300.bin', binary=True)\n\n# calculate word mover's distances between corporate and city initiatives to judge similarity\n# this takes 20 mins approx\ntqdm.pandas()\nemissions['wmd_distance'] = emissions.progress_apply(lambda x: model.wmdistance(x['city_initiative'], x['corp_initiative']), axis=1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# replacing infinities with 4.9 as the max distance\nemissions['wmd_distance'] = emissions['wmd_distance'].replace(np.inf, 4.9)\n\n# the mean of the WMD scores grouped by cities and organizations subracted from 5 gives us the SERO score\ncollab_frame['SERO Score'] = (5 - emissions.groupby(['hq_country', 'cleaned_address_city', 'organization', 'account_number'])['wmd_distance'].mean()) \\\n                            .reset_index([0,1], drop=True)\n\ncollab_frame.sort_values('SERO Score', inplace=True)\ntemp = collab_frame.dropna(subset=['SERO Score'])[-25:].reset_index()\n\nfig, ax = plt.subplots()\nax2 = ax.twinx()\n\ntemp.plot.barh(x='organization', y='SERO Score', ax=ax, legend=False, xlabel='Organization');\n\ntemp.plot.barh(x='cleaned_address_city', y='SERO Score', ax=ax2, legend=False,  color=[grey]*23 + palette[:2],\n               title='City-Corporate SERO Scores', xlabel='City');","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now that we have all the components for the metric we can finally move on to calculating it."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# calculating Collaboration Strength\ncollab_frame['Collaboration Strength'] = collab_frame['Shared Risks'] * collab_frame['Opportunities'] * collab_frame['SERO Score']\n\ncollab_frame.sort_values('Collaboration Strength', inplace=True)\ntemp = collab_frame.dropna(subset=['Collaboration Strength'])[-25:].reset_index()\n\nfig, ax = plt.subplots()\nax2 = ax.twinx()\n\ntemp.plot.barh(x='organization', y='Collaboration Strength', ax=ax, legend=False, xlabel='Organization');\n\ntemp.plot.barh(x='cleaned_address_city', y='Collaboration Strength', ax=ax2, legend=False,  color=[grey]*23 + palette[:2],\n               title='Top 20 City/Corporate pair by Collaboration Strength', xlabel='City');","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"This shows us there exist strong incentives for the possibilities of city-corporate collaborations, allowing cities and corporations to tackle the challenges that affect both adversely. \n\nWith more stakeholders, cities have a greater responsibility towards making more positive changes, even so, collaboration efforts should not be limited to those between corporates and cities. Although in this section we do not explore business-business collaboration, the approaches discussed here for city-corporate collaboration might just as easily be applied to businesses looking to work together. \n\nWith the data showing issues like inequality of resources, inequality of climate exposure and health impacts, unequal consumption, and a lack of emphasis on social issues by corporates, there is an urgent need for finding ever more avenues for city-city and business-business collaborations. 21st-century issues like the “Interwovenness of Climate Change and Inequality” require collective action and mass mobilisation. Tackling this issue is the duty of every individual, government and organisation, making it essential for all of humanity to work together in creating a new future that is equitable and habitable. \n\n\n\n<h1 style=\"color: #FFFFFF\" id=\"#key-takeaways\">9 Key Takeaways</h1>\n\nWhen analysing the CDP Questionnaire we noticed sections to which we had no access. However, we believe the responses could be used to make more environment and economic centric metrics.\n- Questions C2.1a, C2.2a, C2.2b, C3.3a from the CDP Climate Change Questionnaire - Calculate liability from fines as a percentage of expected financial impact and use that to measure climate change insurance.\n- Section C11: Carbon pricing from the CDP Climate Change Questionnaire - Assess the effects of internal pricing on carbon emissions for corporates and measure carbon credits incentive.\n \n\n- These metrics act as a risk assessment toolkit, while also highlighting the recurring issue of inequality in various contexts, showing that vulnerable populations are the most prone to climate change, and poorer cities not equipped to make economic transitions to adapt.\n- The role of businesses has changed over the last decades. Businesses control the significant flow of resources, from food to oil. As a result, it is imperative to hold businesses accountable and engage them in solving the global climate crisis and related inequality issues.\n- The transition to sustainability lies in the ability to collaborate and uplift vulnerable populations, cities and businesses. Climate change is an opportunity to push human innovation and challenge the status quo. It is a portal, a gateway between one world and the next.\n\nThank You!\n"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}