Unlocking
Climate
Solutions\n
![\"wikimedia\"](\"https://upload.wikimedia.org/wikipedia/commons/a/a0/Climate_change_mitigation_icon.png\"){kind=icon}
| CDP: Unlocking Climate Solutions\n | \n \n | \n
CDP is an organization that investigates global environmental disclosure system. Every year, many cities and companies are involved in investigations into the climate, its consequences and their interest in it. CDP wonders how companies and cities are dealing with the climate emergency. By extracting information from the various treaties and their results, we hope to improve the climate transition.
\nIn this section, we focus on cities. We recall that the objective of this work is to find correlations between the recorded values in order to interpret the numerous responses to the questionnaires. To do this, we first examine the numerical values and categorical values that can easily be transformed into numerical values. We first try to extract KPI from those data that incorporate city answers only. Then, in section 1.2, we analyze information from the textual responses. There are methods for extracting topics that are useful to get an overview of the reflections.
\n\nOur first interest go for the easy-to-interpret responses, i.e., the data that are either numerical either categorical. Indeed the questionnaries contain a lot of textual information that require deep data processing. With numercal values, we aim at giving first coorelation between important climate keys.
"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"cities_2020.head(2)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":false,"trusted":true},"cell_type":"code","source":"cities_2020.shape","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"A line is an answer of a city to a question. There are 869313 rows. Not all cities respond to all questions. The next code shows the mean number of questions that haven answered but also the minimal and maximal. "},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"cities_2020[\"Account Number\"].value_counts().describe()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"In order to get coorelations between the values. We should group the value per `Account Number`. Remember in this first work, we use only the numerical and categorical answers. "},{"metadata":{},"cell_type":"markdown","source":"There are 4 mains methds: State or region vulnerability and risk assessment methodology, IPCC climate change impact assessment guidance, Proprietary Methodology, and. Agency specific vulnerability and risk assessment methodology which are very vague and similar. This answer might not be interested to consider.
\n\nA interested question about risk and vulnerability assessment is the reason of cities to take the decision to investigate in it or not. Below we see the different reasons. We notice that the answers are very varied.
"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# Identify and describe the factors that most greatly affect your city’s ability to adapt to climate change and indicate how those factors either support or challenge this ability.\ncities_2020_risk_factor_ability_adapt = extract_answer(cities_2020,'2.2','Factor Affect Climate Adaptation', condition=(cities_2020['Column Number'] == 1))\ncities_2020_risk_factor_ability_adapt['Factor Affect Climate Adaptation'].replace('^(Other).*','Other',regex=True, inplace=True)\ncities_2020_risk_factor_ability_adapt=cities_2020_risk_factor_ability_adapt.groupby('Factor Affect Climate Adaptation').filter(lambda x : len(x)>3)\n\ndf=cities_2020_risk_factor_ability_adapt['Factor Affect Climate Adaptation'].value_counts().to_frame().transpose()\nfig = plt.figure(figsize=(22,12))\nax1 = fig.add_subplot()\ndf.plot(kind='barh',stacked=True,legend=False, color=colors, ax=ax1, grid=False, width=0.04)\nax1.set_ylabel('')\nfor p in range(0,len(ax1.patches)):\n b = ax1.patches[p].get_bbox()\n ax1.annotate(df.columns[p] , ((b.x0 + b.x1)/2 - 0.2 , b.y1 + 0.01),rotation=-280,fontsize=13)\nax1.set_xticklabels([])\nax1.set_yticklabels([])\nax1.set_frame_on(False)\nax1.tick_params(tick1On=False)\nplt.title(\"Factors that Affect Ability to Adapt\",fontsize=18)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n\nI first remove the columns that have more than 100 lines with Nan values. Then I remove the rows that still contains values and obtain a nice sub-set of 421 cities and 80 features
"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# count NaN rows\ncount = all_num_2020.isna().sum()\n\n# remove columns that have more than 100 blanks\ncols_to_remove = [x for x in all_num_2020.columns if count[x]>100]\nall_num_2020_remove_Nan_col = all_num_2020.drop(cols_to_remove,axis=1)\n\n# now the lines\nall_num_2020_no_Nan = all_num_2020_remove_Nan_col.dropna()\nall_num_2020_no_Nan.shape","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# normalizing data\nscaler = StandardScaler()\nscaled_features = scaler.fit_transform(all_num_2020_no_Nan)\n\ndbs = {} \nfor k in range(2, 10):\n kmeans = KMeans(n_clusters=k,random_state=12).fit(scaled_features)\n dbs[k] = davies_bouldin_score(scaled_features,kmeans.labels_)\n\nfig = plt.figure(figsize=(13,6))\nax1 = fig.add_subplot(121)\nax2 = fig.add_subplot(122)\n\nax1.plot(list(dbs.keys()), list(dbs.values()))\nax1.set_title(\"Chart to find the best number of clusters\")\n\n# the best is for random_state=17\nkmeans_noNan = KMeans(n_clusters=3,random_state=12).fit(scaled_features)\nplot_pca_of_my_kmeans(scaled_features,kmeans_noNan,ax2)\nax2.set_title( \"Chart to show the new clusters\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"I tried different random_state to get 2 clusters but most scores indicate to choose 7 clusters. As I prefer to have least clusters to interpret them, I choose 3 which is not so bad for a random_state to 12. Let go back to the polygone chart and figures the differences! "},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# what are the top differences between the three centroids:\ndifference_between_centroids = pd.Series(abs(kmeans_noNan.cluster_centers_[0] - kmeans_noNan.cluster_centers_[1])+\\\n abs(kmeans_noNan.cluster_centers_[0] - kmeans_noNan.cluster_centers_[2])+\\\n abs(kmeans_noNan.cluster_centers_[2] - kmeans_noNan.cluster_centers_[1]))\n\ndifference_between_centroids.sort_values(ascending=False, inplace=True)\n\n# a lot of copy/paste code...hhh, sorry, I will work better if you hire me, I promise\nall_num_2020_reduced_c1 = all_num_2020_no_Nan.iloc[np.where(kmeans_noNan.labels_ == 0)]\nall_num_2020_reduced_c1 = all_num_2020_reduced_c1.astype(float).mean(axis=0)\n\nall_num_2020_reduced_c2 = all_num_2020_no_Nan.iloc[np.where(kmeans_noNan.labels_ == 1)]\nall_num_2020_reduced_c2 = all_num_2020_reduced_c2.astype(float).mean(axis=0)\n\nall_num_2020_reduced_c3 = all_num_2020_no_Nan.iloc[np.where(kmeans_noNan.labels_ == 2)]\nall_num_2020_reduced_c3 = all_num_2020_reduced_c3.astype(float).mean(axis=0)\n \ntop_C1 = give_top_n(15,difference_between_centroids, all_num_2020_reduced_c1)\ntop_C2 = give_top_n(15,difference_between_centroids, all_num_2020_reduced_c2)\ntop_C3 = give_top_n(15,difference_between_centroids, all_num_2020_reduced_c3)\n\n# and we plot it !!\ntheta = radar_factory(15, 'polygon')\n \nfig = plt.figure(figsize=(10,10))\nax = fig.add_subplot(1, 1, 1, projection='radar')\n\ncolors_2 = [colors[0*2+1],colors[1*2+1],colors[2*2+1]]\ni=0\nfor d in [top_C2,top_C1,top_C3]:\n ax.plot(theta, d, color= colors_2[i] )\n ax.fill(theta, d, color= colors_2[i], alpha=0.25)\n i+=1\nax.set_varlabels([x for x in top_C2.index])\nplt.title(\"Top 15 features that separates the clusters\", fontsize=16)\nplt.show()\n\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"This chart shows the 15 main differences and similarities between the clusters. For instance, we remark that the blue cluster is mainly the cities that detailed several factors that affect their adapation while the red cluster represent the one that don't have any target for GHG reduction and don't have to answer to the External Control Reduction. Finaly the purple cluster has a cite-wide inventory to report but did not detail factor that affects the adapatations. "},{"metadata":{},"cell_type":"markdown","source":"\nIn the previous section, we work with the dataset to explore it. We aims at finding solution to the compete problem, i.e., can we extract KPI in the dataset ?
\n\n