{"cells":[{"metadata":{},"cell_type":"markdown","source":"![Image](https://1et31t3azwc3jxcfm1s69wb8-wpengine.netdna-ssl.com/wp-content/uploads/2018/04/Indian-apparel-industry.gif)"},{"metadata":{},"cell_type":"markdown","source":"**Little About the Domain**"},{"metadata":{},"cell_type":"markdown","source":"Foreign trade Includes Import and Export.\n**Importing** means buying foreign goods and services by citizens, businesses and government of a country. No matter, how they are sent to the country. They can be shipped, sent it by e-mail, or even hand carried in personal luggage on a plane. A country importing more than it’s export, runs a **trade deficit**. Whereas, a country importing less than it’s exports, create a **trade surplus**.\n\n**Exporting** means goods and services which are produced in one country are purchased in another country. It is produced domestically and sold to someone in a foreign country. Most countries want to increase their exports as it increases the GDP of the country.\n\nIn India At the level of Central Government it is administered by the Ministry of Commerce and Industry.\n\nPrior to the 1991 economic liberalisation,India was a closed economy due to the average tariffs exceeding 200 percent and the extensive quantitative restrictions on imports. Foreign investment was strictly restricted to only allow Indian ownership of businesses. Since the liberalisation, India's economy has improved mainly due to increased foreign trade\n\n[More on Quora](https://www.quora.com/What-is-import-and-export)\n[More on Wikis](https://en.wikipedia.org/wiki/Foreign_trade_of_India)"},{"metadata":{},"cell_type":"markdown","source":"**Loading Libraries**\n\nwe will import libraries for data processing and preparing charts "},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# charts\nimport seaborn as sns \nimport matplotlib.pyplot as plt\nimport squarify #TreeMap\n\n# import graph objects as \"go\"\n\nimport plotly.graph_objs as go\n\n%matplotlib inline\n\n#ignore warning \nimport warnings\nwarnings.filterwarnings(\"ignore\")\n\n# Input data files are available in the \"../input/\" directory.\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Data Loading**\n\nwe need to load two files one for import and other for export . The Files contain import and export data from 2010 to 2018."},{"metadata":{"trusted":true},"cell_type":"code","source":"data_import = pd.read_csv(\"/kaggle/input/india-trade-data/2018-2010_import.csv\")\ndata_export = pd.read_csv(\"/kaggle/input/india-trade-data/2018-2010_export.csv\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Data Sneak Peek**\n\nwe will take a quick look on what data we have . "},{"metadata":{"trusted":true},"cell_type":"code","source":"data_export.head(5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"data_import.head(5)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"In both the File we have 5 columns each.\n* HSCode - HS stands for Harmonized System. It was developed by the WCO (World Customs Organization) as a multipurpose international product nomenclature that describes the type of good that is shipped\nHS Code Structure\n\n    The HS code can be described as follows:\n\n    It is a six-digit identification code.\n    It has 5000 commodity groups.\n    Those groups have 99 chapters.\n    Those chapters have 21 sections.\n    It’s arranged in a legal and logical structure.\n    Well-defined rules support it to realize uniform classification worldwide.\n    \n   the HSCode in column is 99 chapters \n   \n   [Reference](https://www.tradefinanceglobal.com/freight-forwarding/what-is-an-hs-code/)\n   [HSCode List  ](http://www.cybex.in/HS-Codes/Default.aspx)\n   \n* Commodity - the column contain chapter wise commodity category. In each commodity Category there are various commodities.\n\nA **commodity** is an economic good or service that has full or substantial fungibility: that is, the market treats instances of the good as equivalent or nearly so with no regard to who produced them. \n\n[Reference](https://en.wikipedia.org/wiki/Commodity)\n\n* Value - values for export and import of commodities in million US $.\n* Country - Country Imported From/ Exported To\n* Year - Year in which comodities where Imported/Exported which is in between 2010 to 2018."},{"metadata":{"trusted":true},"cell_type":"code","source":"data_export.describe()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* HSCode shows data is under 1 to 99 which is correct as we discussed above.\n* Value is showing a huge outlier as 75 % data is below 3.7 and maximum is 19805 seems some items category are very expensive .we will research it on this in later.also min is zero as some export may be two small to roundoff in two decimal.\n* Year is under 2010 to 2018 Perfect."},{"metadata":{"trusted":true},"cell_type":"code","source":"data_import.describe()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* HSCode shows data is under 1 to 99 which is correct as we discussed above.\n* Value is showing a huge outlier as 75 % data is below 4.9 and maximum is 32781 seems some items category are very expensive .also min is zero as some import may be two small. \n* Year is under 2010 to 2018 Perfect.\n\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"data_export.info()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"data_import.info()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* Export files  have 137023 rows of data\n* Import files around have 67799 rows of data\n* Value contain null value for both import and export\n"},{"metadata":{},"cell_type":"markdown","source":"**Code Cleanup**\n\nso there is need of code cleanup to better visualize the data.\nlets find out the key area to cleaup\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"data_import.isnull().sum()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"data_import[data_import.value==0].head(5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"data_import[data_import.country == \"UNSPECIFIED\"].head(5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Duplicate imports : \"+str(data_import.duplicated().sum()))\nprint(\"Duplicate exports : \"+str(data_export.duplicated().sum()))\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"From the Above data check we find out columns to cleanup.\n* Value column  has Null values.\n* Value Column has zero value.\n* Country column has unspecified value.\n* Duplicate imports rows\n\nthere can be various way to handle it but for now we are deleting the rows .\nDiscalmer:- As we are deleting rows some of the analysis may be imapcted but it is ok for educational purpose."},{"metadata":{"trusted":true},"cell_type":"code","source":"def cleanup(data_df):\n    #setting country UNSPECIFIED to nan\n    data_df['country']= data_df['country'].apply(lambda x : np.NaN if x == \"UNSPECIFIED\" else x)\n    #ignoring where import value is 0 . \n    data_df = data_df[data_df.value!=0]\n    data_df.dropna(inplace=True)\n    data_df.year = pd.Categorical(data_df.year)\n    data_df.drop_duplicates(keep=\"first\",inplace=True)\n    return data_df","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"data_import = cleanup(data_import)\ndata_export = cleanup(data_export)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"data_import.isnull().sum()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":" ** Commodity Analysis**"},{"metadata":{},"cell_type":"markdown","source":"* Commodity Import Count"},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Import Commodity Count : \"+str(len(data_import['Commodity'].unique())))\nprint(\"Export Commodity Count : \"+str(len(data_export['Commodity'].unique())))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"So why is there just 98 Commodity when there 99 chapters.(This is not deleted in code cleanup)\nSo to findout where is the missing commodity. I printed all HSCode and find out that 77 is missing.\n* HSCode 77 is actually reserved for Possible Future Use.\n\n[HSCode-77 ](http://www.cybex.in/HS-Codes/Reserved-Possible-Future-Use-Chapter-77.aspx)"},{"metadata":{},"cell_type":"markdown","source":"* Commodity count based on different import (country/year)\nLets count the most popular import Commodity . popular just as no of transactions(country/year) are more for this category."},{"metadata":{"trusted":true},"cell_type":"code","source":"df = pd.DataFrame(data_import['Commodity'].value_counts())\ndf.head(20)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"No of Country were we are importing Comodities are \"+str(len(data_import['country'].unique())))\nprint(\"No of Country were we are Exporting Comodities are \"+str(len(data_export['country'].unique())))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"So India is doing Trade  with around 246 Countries. \n\nAccording http://www.world-country.com/ there are 247 countries and territories . 192 are only recognised by United Nation.\n\nWe cannot ignored the possibilty that data can be in short form or same country represented multiple time different way."},{"metadata":{},"cell_type":"markdown","source":" **Import And Export Year Wise**"},{"metadata":{"trusted":true},"cell_type":"code","source":"df3 = data_import.groupby('year').agg({'value':'sum'})\n\ndf4 = data_export.groupby('year').agg({'value':'sum'})\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df3['deficit'] = df4.value - df3.value\ndf3","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"\n# create trace1 \ntrace1 = go.Bar(\n                x = df3.index,\n                y = df3.value,\n                name = \"Import\",\n                marker = dict(color = 'rgba(0,191,255, 1)',\n                             line=dict(color='rgb(0,0,0)',width=1.5)),\n                text = df3.value)\n# create trace2 \ntrace2 = go.Bar(\n                x = df4.index,\n                y = df4.value,\n                name = \"Export\",\n                marker = dict(color = 'rgba(1, 255, 130, 1)',\n                              line=dict(color='rgb(0,0,0)',width=1.5)),\n                text = df4.value)\n\ntrace3 = go.Bar(\n                x = df3.index,\n                y = df3.deficit,\n                name = \"Trade Deficit\",\n                marker = dict(color = 'rgba(220, 20, 60, 1)',\n                              line=dict(color='rgb(0,0,0)',width=1.5)),\n                text = df3.deficit)\n\n\ndata = [trace1, trace2, trace3]\nlayout = go.Layout(barmode = \"group\")\nfig = go.Figure(data = data, layout = layout)\nfig.update_layout(\n    title=go.layout.Title(\n        text=\"Yearwise Import/Export/Trade deficit\",\n        xref=\"paper\",\n        x=0\n    ),\n    xaxis=go.layout.XAxis(\n        title=go.layout.xaxis.Title(\n            text=\"Year\",\n            font=dict(\n                family=\"Courier New, monospace\",\n                size=18,\n                color=\"#7f7f7f\"\n            )\n        )\n    ),\n    yaxis=go.layout.YAxis(\n        title=go.layout.yaxis.Title(\n            text=\"Value\",\n            font=dict(\n                family=\"Courier New, monospace\",\n                size=18,\n                color=\"#7f7f7f\"\n            )\n        )\n    )\n)\n\n\n\n\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* Import is always more than the export creating a trade defecit which we can see in red bar graph.\n* In 2011 2012 showing a huge huge trade deficit and after which it gradually decreases till 2016 and then increased in 2017 and 2018."},{"metadata":{},"cell_type":"markdown","source":" **Import And Export Country Wise**"},{"metadata":{"trusted":true},"cell_type":"code","source":"df5 = data_import.groupby('country').agg({'value':'sum'})\ndf5 = df5.sort_values(by='value', ascending = False)\ndf5 = df5[:10]\n\ndf6 = data_export.groupby('country').agg({'value':'sum'})\ndf6 = df6.sort_values(by='value', ascending = False)\ndf6 = df6[:10]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sns.set(rc={'figure.figsize':(15,6)})\nax1 = plt.subplot(121)\n\nsns.barplot(df5.value,df5.index).set_title('Country Wise Import')\n\nax2 = plt.subplot(122)\nsns.barplot(df6.value,df6.index).set_title('Country Wise Export')\nplt.tight_layout()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* China has biggest market in india  followed by UAE,Saudi Arabia and USA\n* USA is our biggest importer followed by UAE and China Republic.\n\n**Trade Defecit/Surplus Top 5 country  **\n* China - very Huge Trade Deficit\n* UAE - little Trade Surplus\n* Saudi Arabia- Huge Trade Deficit\n* USA - little Trade Surplus\n* Swizerland - not making even in the top export graph showing sign of Huge Trade Deficit."},{"metadata":{},"cell_type":"markdown","source":" **Import And Export Year Wise Trend**"},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = go.Figure()\n# Create and style traces\nfig.add_trace(go.Scatter(x=df3.index, y=df3.value, name='Import',mode='lines+markers',\n                         line=dict(color='firebrick', width=4)))\nfig.add_trace(go.Scatter(x=df4.index, y=df4.value, name = 'Export',mode='lines+markers',\n                         line=dict(color='royalblue', width=4)))\nfig.update_layout(\n    title=go.layout.Title(\n        text=\"Yearwise Import/Export\",\n        xref=\"paper\",\n        x=0\n    ),\n    xaxis=go.layout.XAxis(\n        title=go.layout.xaxis.Title(\n            text=\"Year\",\n            font=dict(\n                family=\"Courier New, monospace\",\n                size=18,\n                color=\"#7f7f7f\"\n            )\n        )\n    ),\n    yaxis=go.layout.YAxis(\n        title=go.layout.yaxis.Title(\n            text=\"Value\",\n            font=dict(\n                family=\"Courier New, monospace\",\n                size=18,\n                color=\"#7f7f7f\"\n            )\n        )\n    )\n)\n\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* slowdown in trade between 2014 - 2015 .\n* In 2013 it shows export was on increase but the import was decreasing.\n* Export shows Sideway trend  till 2014 after sharp decline in 2015 the export shows an upward trend.\n* Import increased 2010 to 2011 then it shows a downward trend till 2015 and later sharp upward trend "},{"metadata":{"trusted":true},"cell_type":"code","source":"df3 = data_import.groupby('Commodity').agg({'value':'sum'})\ndf3 = df3.sort_values(by='value', ascending = False)\ndf3 = df3[:10]\n\ndf4 = data_export.groupby('Commodity').agg({'value':'sum'})\ndf4 = df4.sort_values(by='value', ascending = False)\ndf4 = df4[:10]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sns.set(rc={'figure.figsize':(15,10)})\n#ax1 = plt.subplot(121)\nsns.barplot(df3.value,df3.index).set_title('Commodity Wise Import')\nplt.show()\n#ax2 = plt.subplot(122)\nsns.barplot(df4.value,df4.index).set_title('Commodity Wise Import')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* Top exported categories are also the top imoported categories but there is huge trade deficit category wise. \n* Vehicle Other Than railway...and Pharmaceutical product HScode Chapters shows a trade surplus \n"},{"metadata":{},"cell_type":"markdown","source":"1. **let's Analyse Expensive Import **\n\nAs we have seen there is huge difference in distribution we will analyse some of the expensive imports"},{"metadata":{"trusted":true},"cell_type":"code","source":"expensive_import = data_import[data_import.value>1000]\nexpensive_import.head(10)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"** Import Value Vs HSCode(Commodity Code)**"},{"metadata":{"trusted":true},"cell_type":"code","source":"#fig, ax = plt.subplots(1,1,figsize=(18,6)) \nplt.figure(figsize=(20,9))\n#plt.rcParams['figure.figsize']=(23,10)\n# the size of A4 paper\n#fig.set_size_inches(11.7, 8.27)\nax = sns.boxplot(x=\"HSCode\", y=\"value\", data=expensive_import).set_title('Expensive Imports HsCode distribution')\nplt.show()\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* HSCode Chapter 27, 71 tops the expensive  import as already seen (Mineral Fuels and expensive jewellery)\n* HSCode Chapter 15,29, 84, 85  also has a expensive imports."},{"metadata":{"trusted":true},"cell_type":"code","source":"df =expensive_import.groupby(['HSCode']).agg({'value': 'sum'})\ndf = df.sort_values(by='value')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":" \nvalue=np.array(df)\ncommodityCode=df.index\nplt.style.use('ggplot')\nplt.rcParams['figure.figsize'] = (10.0, 6.0)\nsquarify.plot(sizes=value, label=commodityCode, alpha=.7 )\nplt.axis('off')\nplt.title(\"Expensive Imports HsCode Share\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* The share of the HSCode category 27, 71, 85, 84 in expensive Category has most valuable imports"},{"metadata":{},"cell_type":"markdown","source":"**Country Analysis** "},{"metadata":{"trusted":true},"cell_type":"code","source":"len(expensive_import['country'].unique())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df1 = expensive_import.groupby(['country']).agg({'value': 'sum'})\ndf1 = df1.sort_values(by='value')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"value=np.array(df1)\ncountry=df1.index\nplt.style.use('ggplot')\nplt.rcParams['figure.figsize'] = (10.0, 10.0)\nsquarify.plot(sizes=value, label=country, alpha=.7 )\nplt.title(\"Expensive Imports Countrywise Share\")\nplt.axis('off')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* Country Wise for Expensive Items China, Saudi Arab, UAE, Swizerland has most share followed by USA, Iraq "},{"metadata":{},"cell_type":"markdown","source":"**Conclusive Comments**\n* Need of the Hour is to reduce the trade deficit\n* New Initiative taken by Goverment as \"Skill India\" , \"Make In India\", \"Startup India\" can help to boost the Export if the work is implemented on ground reality. \n* Bilateral ties between countries helps to reduce export duty which help the local company compete in global market.\n* As India Primarily is a Agricultural Country training and Guiding and providing export quality crops/medicinal plant can help to boost agricultural export.\n"},{"metadata":{},"cell_type":"markdown","source":"   **Thank you for taking Out Time to read this. Please Upvote Comment to keep me motivated**"},{"metadata":{},"cell_type":"markdown","source":"![](http://pluspng.com/img-png/animated-thank-you-png-for-powerpoint-copyright-2018-animations-media-960.gif)"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}