{"cells":[{"metadata":{},"cell_type":"markdown","source":"## Goal: Make Residual Network For Digit Recognizer Competition"},{"metadata":{},"cell_type":"markdown","source":"> Note: To run this you'll need to set the accelerator to tpu"},{"metadata":{},"cell_type":"markdown","source":"### In this notebook, I wanted to do the following:\n1. Create a neural network that used a ResNet's residuals/skip connections idea\n2. Employ the best practices when designing the model\n3. Train the model on a tpu\n4. Increase the complexity of the dataset with data augmentation"},{"metadata":{},"cell_type":"markdown","source":"#### I'll try to include everything I learned as well as good practices I found"},{"metadata":{},"cell_type":"markdown","source":"### Update Tensorflow Version on Kaggle"},{"metadata":{},"cell_type":"markdown","source":"#### The default kaggle environment didn't have tensorflow 2.4 installed which you need to have this: *tf.distribute.TPUStrategy*"},{"metadata":{"trusted":true},"cell_type":"code","source":"!pip install -U tensorflow==2.4\n\n!pip install cloud-tpu-client\nimport tensorflow as tf \nfrom cloud_tpu_client import Client\nprint(tf.__version__)\nClient().configure_tpu_version(tf.__version__, restart_type='ifNeeded')\n\n#!pip install efficientnet\n\nprint(tf.__version__)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Import packages"},{"metadata":{"trusted":true},"cell_type":"code","source":"import os\nimport numpy as np\nimport pandas as pd\nimport tensorflow as tf\n\nfrom tensorflow import keras\nfrom tensorflow.keras import layers\nfrom tensorflow.keras import callbacks\nfrom sklearn.preprocessing import StandardScaler, OneHotEncoder\nfrom sklearn.compose import make_column_transformer\nfrom sklearn.preprocessing import OneHotEncoder","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Retrieve training set"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df = pd.read_csv('/kaggle/input/digit-recognizer/train.csv')\n\n#Seperating features from label\nX = train_df.copy()\ny = X.pop('label')\n\ntrain_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Quick One Hot Encoding"},{"metadata":{},"cell_type":"markdown","source":"The label \"0-9\" is a number but we don't want our model to consider it as an ordinal value but like a categorical value. So we'll use one hot encoding on our labels."},{"metadata":{"trusted":true},"cell_type":"code","source":"mlb = OneHotEncoder()\ny_d = mlb.fit_transform(y.to_frame().values)\ny_df = pd.DataFrame.sparse.from_spmatrix(y_d)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Keras Subclassing, Residual Module"},{"metadata":{},"cell_type":"markdown","source":"Here is the residual module. \n\nExplanation: The idea behind skip connections is that you add the current layer with it's previous layer. When you go back and do backpropgation the previous layers receive the same updates as the later layer. This is good because as more layers get added to the top of a neural network (it starts on the bottom) the bottom layers receive less updates. \n\nA downside to this method is that you can't downsample the image until you add the layers together"},{"metadata":{},"cell_type":"markdown","source":"![image.png](attachment:image.png)\n\nImage from here: https://towardsdatascience.com/residual-blocks-building-blocks-of-resnet-fd90ca15d6ec","attachments":{"image.png":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAjoAAAFJCAYAAACSFocyAAAgAElEQVR4AeydB1QU5/f333MAW+wpthhQVGyAvXewYDcao7En9iSaGDTGTmKLPcauUWPvBUs09opgVyz8RBGiUkKTIvBnN9/3zE7Z2WV32V22zM5eztmzZeYp93Nnd77c5z7P8/9Af0SACBABIkAEiAARkCmB/ydTu8gsIkAEiAARIAJEgAiAhA5dBESACBABIkAEiIBsCZDQka1ryTAiQASIABEgAkSAhA5dA0SACBABIkAEiIBsCZDQka1ryTAiQASIABEgAkSAhA5dA0SACBABIkAEiIBsCZDQka1ryTAiQASIABEgAkSAhA5dA0SACBABIkAEiIBsCZDQka1ryTAiQASIABEgAkSAhA5dA0SACBABIkAEiIBsCZDQka1ryTAiQASIABEgAkSAhA5dA0SACBABIkAEiIBsCZDQka1ryTAiQASIABEgAkSAhA5dA0SACBABIkAEiIBsCZDQka1ryTAiQASIABEgAkSAhA5dA0SACBABIkAEiIBsCZDQka1ryTAiQASIABEgAkSAhA5dA0SACBABIkAEiIBsCZDQka1ryTAiQASIABEgAkSAhA5dA0SACBABIkAEiIBsCZDQka1ryTAiQASIABEgAkSAhA5dA0SACBABIkAEiIBsCZDQka1ryTAiQASIABEgAkSAhA5dA0SACBABIkAEiIBsCZDQka1ryTAioJuAQqFAl4AuqF6jGiIiInSfRJ8SASJABGRCgISOTBxJZhABYwncuXMHrm4uqscvc38xthidRwSIABFwSAIkdBzSbdRpexLIfhaMP9ZtxOYtf2Lnzt3Yt28/DuzZjV3b/sTWTRuxYf0OXHqdI+piNl4c/wMbNvyBrX/uwO49+3Bg7x7s3rYVWzZswM7zryA+W1TQKi9Dw0IFoTNz5gyrtEGVEgEiQASkQoCEjlQ8Qf1wGALJR/vC093DwKMBplxPF9mTjJMDPPWe3/D7axCfLSpolZckdKyClSolAkRAogRI6EjUMdQt6RLI/ucKgg8cwMEDe7F2XDMtAdMeUzadwN2EXJEB2Yg+uQo/9fXmzq0G/xFTMX/xCqzdsA1HwxIgPltU0CovSehYBStVSgSIgEQJkNCRqGOoWw5C4N1TbPq8pobYqdVnNcIzNPufG3sSk5sxUaDq6LP8DtKUmsdt+Y6Eji1pU1tEgAjYmwAJHXt7gNp3eAKKfy9gZmvNoazmE47hNRemUaaGYkkAO3TVYdo5JCjsazIJHfvyp9aJABGwLQESOrblTa3JlED2//7E0NpisVMNvZbeQVpmBLYOqquK+DQevQ8vs+0PgISO/X1APSACRMB2BEjo2I41tSRrAkokXw2Cn0aSsjd69GigEjm1+67BQ63hLHvhIKFjL/LULhEgAvYgQELHHtSpTZkSyEbUrhGopyF2PODZbg6uJNp5vEpEnISOCAa9JAJEQPYESOjI3sVkoE0JKFMROq8TqonFju9oHIi25Uo5hi0moWOYDx0lAkRAXgRI6MjLn2SN3Qnk4J+DY9BALHSY1/7zcSPVjlOtRFxI6Ihg0EsiQARkT4CEjuxdTAbajoACiVeC0JERNh4t0VM1nVydoFxvxC5EUTKy7dxBLREBIkAEAJDQocuACFiIQEb4WgyowQibZgg88RrvYg7h6/pqocOspuwfdA3Jdg7sUETHQg6naogAEXAIAiR0HMJN1EmpE8iJOYivGzCipi6GbonAO1WHlUi/uxy9PMVipw6G/xkJewZ2SOhI/Wqi/hEBImBJAiR0LEmT6nJKAop/LyKoPSNmqqLbwlBopuLkIvb4d2ihkbPTFrMu/At7zcMioeOUlykZTQSclgAJHad1PRluCQI58VewtHcN1Vo59UcfFlZD1qw7E4/W9EMtsdipOQBbIti4j+a51n8nFjolS5VAxUoVTH6UKl0SRYoWRrnyH+ET98rwqOKOqp5VUL1GNXjVrIFatWuiTt3a8PapC996PqjfoB4aNmqAxk0aoXnzZvDz90Ov3r0weMggjB03BpMnByIoaA6WLluK9RvWY9euXQgOPooLF84j7GYYnjx5gn/++QepqalQKOwlEa3vG2qBCBAByxMgoWN5plSjrAlk48m6Eejarg1aN/FBDbF4ca8L/4mnkaiVg/Pu4Wp81bUNfDXO9YBnrdYI6BKAbgH9EXT9rc2oHT9xHK5uLg79KF7iPZSvUE4lrBo1boguAV0wdNgQBAb+gIW/LsTmzZtx7FgwboTewIsXL5Cebsv94W3mSmqICBABIwiQ0DECEp1CBNQEMnBrelNuE8+q8KrhhdrVqwnr5lTruQdxWkInPWQKmgoipyq8qnuhTo0a8KrC5+5Ux+CTKeomrPxKHNGpULG8KtrCRFxMeVT+5GOUfb8MatbyUkVtmOgN87qGV3VUq+6piu4wUR7mvEofVwTTDhP9+fCjD1CmbGkUKuxmc6HFiKMqVT1UEaV+n/XDpEnfY/mK5Th48ABu3rqJ+Ph4K5On6okAEbAHARI69qBObRIBOxIQC52ZM2fYrSfv3r1DQkICnj9/jvv37+Pqtas4ffoUDhzYj61bt+L3Vb9jwcIFmD59GiZOnIAvvxqB/p/3R0DXALRp01olzJihsg8+fN9ioum94sVUgq1jp44YOeorBP0chO3bt4Nh9vat7aJudnMKNUwEZEiAhI4MnUomEQFDBKQidAz10dRj//d//4c3b97g3r17OHv2jCrH57eVv2HGjOkYM3Y0+nzaB61bt1LlDzH5ReYO3X1cuRI6+HXA+PHjwNR/6tRfiIqKwn///Wdql+l8IkAEbESAhI6NQFMzREAqBOQodExlm5ycrBJFTB7PqtWr8OOPUzDwiwFo1aqlKrnarZCrSWKIGRZjhv6GDB2sSqi+ePGCKnHa1H7R+USACFieAAkdyzOlGomApAmQ0MnfPUyEiElivnjpIjZt2qSaFdazV09VRMiU/CImZ+nzAZ+rEqT//vs0EhMT82+cziACRMCiBEjoWBQnVUYEpE+AhE7BfJSTk4NHjx7h0KGDmL9gPoaPGKZKcGaSrI0ZEmMSoj/r/xlW/LYCt2/fhlKplb1esO5RaSJABLQIkNDRAkJviYDcCZDQsZ6HmcRqJpmaSaBmprwzM83yEz/M7LVu3bupoj7Xrl8DE02iPyJABCxHgISO5VhSTUTAIQiQ0LGtm2JiYnD06BHMmTMbPXr2AJPQbEj8MPk+zIKKzAKKV65eoQUSbesuak2GBEjoyNCpZBIRMESAhI4hOrY5xuT/bNu2DaNGj1RNZzckfJi1hwYN/gI7d+6kHB/buIdakRkBEjoycyiZQwTyI0BCJz9Ctj8eGxuLffv24dtvv0G9+r56Iz7MbDBmZti8+fNw9+5d23eUWiQCDkiAhI4DOo26TAQKQoCETkHo2aYsM/39yJHDqjWAmL3E9EV8mGMTJnwLJreH/ogAEdBNgISObi70KRGQLQESOo7nWiZ6w0RxmGiOvjV+PKtVxbRpP6lWmXY8C6nHRMB6BEjoWI8t1UwEJEmAhI4k3WJ0p/7991/s2LFDlbfz/gdldUZ7fHy9VcKIyQWiPyLg7ARI6Dj7FUD2Ox0BEjrycTmzpg8zo2vAwM/BzNbSNcTFbH3BJD5nZ2fLx3CyhAiYQICEjgmw6FQiIAcCJHTk4MW8NqSnp6tmZnXv0R2FixTKI3qYNX2YrS4oypOXHX0ibwIkdOTtX7KOCOQhQEInDxLZfcBsNbF+w3q0bdsmj+BhcnyYBQqPHz9GqzLLzvNkkC4CJHR0UaHPiIAMCNy5cwfMf/naf/kJHWYn7qvXrmoXo/cOSiA8PBzffPM1SpcplUf0MAnMS5ctRUZGhoNaR90mAvkTIKGTPyM6gwg4JIGHDx+qtiCYOHECDh8+JCw2p0voMPkbly5fwi9zf1Htwr3w14UOaTN1Wj8BRvSuXbdW5zo9H5X7UJW8/PbtW/0V0BEi4KAESOg4qOOo20TAGALMztniBFXfej7w7+gnfFanbm106NAexd4rKnzG3PTS0tKMqZ7OcVACzNYSzGrLRYsVEfzOXCfMLK5Zs2YKothBzaNuEwENAiR0NHDQGyIgLwJMVEcsdIx5TdEceV0Dhqxh9uFiFhx8r3gxjeukVOmSqsTluLg4Q8XpGBFwCAIkdBzCTdRJImA+Ae2ojiGxQ9Ec8zk7cklmC4rJkwNRslQJDcHDvGeGM9+9e+fI5lHfnZwACR0nvwDIfPkTMCWqQ9Ec+V8PhixkZmvNnDkDZcqW1hA8HlXcsWvXLjCJ6vRHBByNAAkdR/MY9ZcImEHAmKgORXPMACvTIikpKapcHe1FCFu0aI6QGyEytZrMkisBEjpy9SzZRQREBIyJ6lA0RwSMXqoIMDk8g4cM0ojuMEOfTCLzP//8Q5SIgEMQIKHjEG6iThKBghPo/3n/PDcsPl+HojkF5yvnGm6E3kDLli00rh9meGvjxo1yNptskwkBEjoycSSZQQTyI/DgwQONGxUvcphniubkR4+OMwR2796NKlU9NK6jzl06Izo6mgARAckSIKEjWddQx4iA5QnoiupQNMfynOVcIzMDKzDwBzBbSfBimVl1mdlygv6IgBQJkNCRoleoT0TASgR0RXUommMl2DKv9nrIdTALTvJih3nu2KkjXr58KXPLyTxHI0BCx9E8Rv0lAgUkII7qUDSngDCdvHhWVhamTv0RhQq7CYKHWV352LFgJydD5kuJAAkdKXmD+kIEbEBAHNWhaI4NgDtBE8z+ad4+dQWxw0R3pk+fBoVC4QTWk4lSJ0BCR+oecrD+MSusXrt+jR4SZ+Dn3wFl3y+DM2fPkK8k7itmaQBH+GM2hh07boyG2PHv6I/4+HhH6D71UcYESOjI2Lm2Nu38+XMoUrSwxg+dePyeXrsQGzdiYM73gBkecpS/P//8E+KFBj9xr6wS047Sf+qn/AiQ0JGfT+1mEROqNudHnMrQzZ+uAcPXgI+vt92+1+Y0fP/+fXjVrCH8HjD/AG3ZssWcqqgMESgwARI6BUZIFfAEpk37Sfhh69W7F4J+DqIHMaBroADXAC8A63rX4b9mDvOcmpqKT/t+KvwmMLbMnTfXYfpPHZUPARI68vGl3S0RC51bt2/ZvT/UASLg6ASYYR9GIDii0OHZM7uf84KNef7mm6+hVCr5w/RMBKxOgISO1RE7TwMkdJzH12SpbQjIQegwpP744w+NKeif9f8MTPIy/REBWxAgoWMLyk7SBgkdJ3E0mWkzAnIROgyw48ePaSQpd+jQHszwFv0RAWsTIKFjbcJOVD8JHSdyNplqEwJyEjoMMGY1ZWaRSn4oq1WrlkhPT7cJS2rEeQmQ0HFe31vcchI6FkdKFTo5AbkJHcadT5480dgYlFlrh1lhmf6IgLUIkNCxFlknrJeEjhM6nUy2KgE5Ch0GWGRkJD6uXEmI7PTo2QP/93//Z1WWVLnzEiCh47y+t7jlJHQsjpQqdHICchU6jFsfPXqEcuU/EsTO5wM+py0jnPx6t5b5JHSsRdYJ6yWh44ROJ5OtSkDOQocBd+fOHTCbgPI5OyO+HG5VnlS5cxIgoeOcfreK1SR0rIKVKnViAnIXOoxrmQTlUqVLCmJn/oL5TuxxMt0aBEjoWIOqk9ZJQsdJHU9mW42AMwgdBp72PnnHjgVbjSlV7HwESOg4n8+tZjEJHauhpYqdlICzCB3GvevWrxOiOmXKlsbjx4+d1OtktqUJkNCxNFEnro+Ejm2dn/tiKwbUc4eHT1+sj8ixbePIxYutn8O3iju8+62HzZu3sbX2as6ZhA7DePz4cYLYYTYFTU5Othd6aldGBEjoyMiZ9jaFhI4BD6TfwuLPGqKKhze6/xKCNAOnGnso8/ZUVHNjdr2ujG+vZxhbzELnZeLWT57sTcn9G1yzdfMWskLq1Tib0GGmmLdv304QO10CutC+WFK/SB2gfyR0HMBJjtJFEjr6PZXz5Fd4q0SJC1x9f8UTCwRgLCF0FNk5MG97RRI6+r1tuSPOJnQYcgkJCajqWUUQOwt/XWg5oFSTUxIgoeOUbreO0SR0DHBNvYKfWlZCmbIV0XzKZVhih5+CCZ00nB9ZUXUzKea3GdEKA33XeYiEjk4sFv7QGYUOg/Du3bso9l5R1fVZtFgR1XsLo6XqnIgACR0ncra1TSWhY23CmvUXTOgkI7jne+x/zfWX4VmuZt35vyOhkz+jgp/hrEKHIbd4yWIhquPj6027nRf8cnLaGkjoOK3rLW84CR3LMzVUIwkdQ3TkccyZhY5SqdTI15k8OVAeTiUrbE6AhI7Nkcu3QUkJnexI7Jk2Al8MGYWgYzHIG7BQIvHKEnw9bBBGTt+DyGwdfsmJwqE5IzF48EjMPx8PzdGdLERf3ITZo3ujQ1Nv1PFtjPY9h2Pa2r8RlamjLkU8LiwaiyFDR2Pe6Tc6+gMoMyIQvGQSvuzXES0aNUDTdl0xYORojBs3lnuMQ+DKK0jkOpJH6OS8wdUdC/H9IH80rt8Y7fuMxrz9D5EqTsJRJuHKiu8xftxX6OrOJDK7wLVsSwwey7Ux/nssv5qowwDtj/KL6CiQ+vgE1v/8DQZ2a4smvnXQoFVn9B0xAQsPP0W6uE/IxZvTczF6yCCMnLYdT3TxUzWfifAtgRg+eAgmrrsH7fznrOgL2DR7JHp3aAyfut5o0r4HRkxbgzO6HYL48wswZuhwTNn2FNnIwNN909G/jS9q+zZDj4nb8UzXNaGNwcrvnVnoMGijoqJQukwp1XXqVsgVFy9dtDJxql6OBEjoyNGrdrJJUkIn5zEW+HI38oZL805/VsZjXwCbA+Dq5oU59/Pe1bLD56KOKoG4BHoejBeSdpVpd7Gqf1UhrM4vXy88ew3FtmdauzFn3sJP1dn+fPLtNa2btBJpt5agBy88+KRlXc+Vx+JKOutgsdAZvXszRtcvorNP1UftRwyv9DKuY6IHx0VX/W4uqBoYasQVZEDo5DzHjq9qo4ie+hlOlXv9hvuCUlHi9Y6OKKY6vywGnNQzpTj5BAZ+oIOhMg13V38GT73tVcewbc+g6RF1/4t22YkLC9uhpEb5VtiVYAQGK5/i7EKHwbtlyxbhuq5W3ZN2OrfyNSfH6knoyNGrdrJJUkIH2XgQVJP9gSzUHOtfasZjkHoKgz5U3/DrL4uA5kSoXET+3ogtXzwAe+O4EERuDHb24/fmqYAuM7bizL1ovIkMxYk1I9HgPa5OnxkIE27kAAwJndRLmFCVK1fzC6w4eQ/RcS9x5/hiDKzpyvbhwwDMWrMR2049E0SSWujwdhSCV+8p+H3XfuxYGYiAyvznZdH/CCfUlGkIP7QGy5bNw6g63PGKfTBzyVIsXbYUS5evxdEn4o7ru5jUQsFVa3q58tUmNGVEQ+k66DFhMbYfv4Q74fdw5fgm/NipAnfTckOL1ZFCZEsZfwC9S7P9KdPvCHTFlJKO90dZlRipgslhfNgnFzE7++J9TqRUDJiOrWfuIfpNJEJPrMaohryYrYuZN8V2ifpfviY+UpX/GH5jpmP25BHo2SMQlyyxBoA+fEZ+TkKHBdW7T2/uunHBL3N/MZIenUYEWAIkdOhKsBgBaQkdIEtYZ6Yw/Ha8ESIyjMEZIRPgzt0cmQiDW8sN0NBCilfY0q6Q6se1iP92vOZ0TvKZ4SivKlcGvXdECzdqHmLa9R9QXXW8BHruj1O3aUDoJJ/4jLuBe+PnB5qRpezw+Wigqu8DDDmjOVdLU+iURIeFNzSGqXIiVqJVEVY8vNd1PzQDFFZMRs6JxrkDZ/FMc3yKRZR+Fd9xos6t9Ra8EoawknFy0PvszaxUDxyIFw5waBMR3K80e7zWTNzlwzPJf2NEBdbGsn12IJqPXAkOuYbAGuzxkr32g9ergEjoMHwLeSPwfKLW8CRfif2eSeiw7F++fIniJdjk+RIliyMmJsZ+TqGWHY4ACR2Hc5l0Oyw1oYOMEHzHDdOU7HVQdKPPRvjcWqqbZlHPT9ghlmIdsStWfXNVxu9DQHHmBlkIbf6I5m6AiTjUszh7s/WZh3BNTcI6JucpFtfjbrwDTiKFd5deoZODxwvrsHV6BkIIVAjlQhHoydZXZ144xE2qhU5hNJsXhjR199nSynjs78bNrPKarhYHqqNWFDp833U+p+HCyHKsvdWn4hYfmAGQdnEkKqpEXTF03qUpTJFwCH3KsBy856s5JB7qgRKqMnUxT7dD8HSJD9ve+5/jL7VD1AseuhWH35oIDbY6u26HD0noqKEHBc1h/ejmggEDP1cfoFdEIB8CJHTyAUSHjScgOaGDdFwex64V4/rBAJzib3KKKKxrxtw0i6HLhvWcoCmFT4OTBGPVUZZmWPOCCxNkXMU3XB5N5fGXkJKTg5w8j3gc7VOC/UFuuAKRfITBgNB5stibPb/qJNwQ3fhVnUk9g8HcEFv9FeqhHuaYWujoWxk5E2GTuYXXPhkn5PawRtpL6GTgxiR31l73CdBY0DkjBN9XYcVMEb+teCWMNioRf7Anl0PTAEuf8oOMGbj6TWW2rk/G4VKKLn/kIP5ob65sfaxQO0QtdD4eg8tc3pNwAUjkBQkdtSPevXunsZAgJSar2dArwwRI6BjmQ0dNICA9oQOknh+Ocqr/+Ctg1CX2bqaM24vOqlya5tjw4iW2ckNUHw0/yy3kl4aLo8qzN9DGy9VrzCQdQpdi7I1YSDxW1a3ns9pBeMiHYPQKHSD9yhhUVtVTHsOC40TDJ7l4ua0HSquO5RUz+QudLNyf48XaUWkUOPM5j1pf6CiSw3Fs5SQM7uWHlg1rwr18GZQuXYJLOnaBq/u3mkIHWbg7i+tvkVbYwI8lKuOwrwcbSXNrvhrPefGIJBwO0J2Ards/NRGkdoha6GjlGJlwyVv9VBI6mogPHNjPXs9uLqjfoB5tD6GJh97pIUBCRw8Y+th0AlIUOkg+iQHcTJ0qP9wAEzBJOT2QTV71no9HObl4vqox++PJ3/BEkQWfXx+rk5QTdqMtL2wq+aBZi+Zooe/Ryh/Dloep97QyIHSQG4X1bdh8IFe3Cmg77HvMCpqO74a05JJkXVDcf3WemWOSFTrKVNxc0VsjB8q1UAmUc6+Cqp4eqKQaEtQldICcxwtRj2PcbPVzVQ6UMnYPAkowYrIw2gnDiMz1mYDdbXmRWQG+zQz4o0VL+A9fhjAhwViUo8P73fRL3uolSOjkRezf0V8QO7t37857An1CBLQIkNDRAkJvzScgSaGDRBzty67D4VprNu5nZ+D6BHa4gxc+OY/mc/tQ1cDMu1nIvj8btVQ325oIEicHJwejV0n2xur+XYgw+8koYgaETsbtIDQqzNRbFh9yycPqiERZNB+zDqFJwjiO0JxUhU7K2a/wMSdWKvWeiyMPkkRJ2+KhK+2IDoDcSKxqxs00a7wMETlKvNnVGe8x9b3XCTv4rHAVhWQc683lTHlMRIh4UpVASd8LEjr6yEj989u3bwtCh1kx+b///pN6l6l/diZAQsfODpBT89IUOkrE7++G4qobry8WP7iLn1XTqj/E0HPcLKasW5jGzczxXfwQ4cvqsT+kXtNwh5/dwzgq6x5m12KFTpGOu0UzeIzwol6hk4TgviVV7dWacx8ZqRG4dGQHNqz7A3tOXMHD15nqmVtazRRY6PTiEpXrLYaQ9qLVhv63+oRCKs4M5qbfM3kzQgSFrykfoQMFYra059bg8cGC8Chs78gOT5XsKZ41xdSXhXuzuaGuov7YrZ5SxTdm4Flf/w0UscMhiujoht69R3dB7Bw8eED3SfQpEeAIkNChS8FiBKQpdADlmx3w43JrmsyaicaM6CnRHQeF+daipNamszGHiyhUCQxVDXWpAYmSm4u2wZpIPilWfYbeV/qETu4zLKvPRYnGn8BrU6oUps/nzd9h+2EoRycVpwZygqTqDwjVToLWawh/QJ9QSMCedlxExndRXgGliMWhz7l28+TosHUrY/eiGxc5q/XNdLQpyvApg37BeVfXYfKbKqlEbGG0XRupHmbku6n3WV//9RawywESOrqx3wi9IQidBg3rU1RHNyb6lCNAQocuBYsRkKrQgSIGf7TlcmCKFlb9QLq13oQY0WhQ6tmhXD5MYS6aUBnfXMs7FpL1YC4aqm6sLijSeBKORolDPgxKBdKencXGufNx+KVItegTOshE2E/cwoZMvYXKwqNWHdRr1BgtWrdHp96D8PX0Jdh1IzbPTbxgEZ1s3J/Dt1sNgVdS2MhRbioS0kVg9F4d+oQCIwYrsTehQvURdEs0nSk7CocnNUJRjl/eZGS+sUQE9y8j3MhUw3gfDcZpftYcfxrznPUAcxuxQtG1aEP8cDRKawVkQJH2DOc2/oL5h1+KGOrrv7hy+78moaPfB527dBaukeDgo/pPpCNOT4CEjtNfApYDIFmhg1y8WNtU+FFkbpw+i56IbnoAko7iU26dFtWNtdJIXMwz7MKwysSDZX7cTCjmBlsYH1XzQUv/TvBr0wQ+7txwkNsH6qExVTEDW0BkPsCiFqwAU+fmcDdvXhS4ucC933o8EemqggkdIPPmFG5xQ6atEqhcswYql3BF9ak3jbgo9AuFrAfz0USVc8TUWxmt+gzE0IHd0KQia1OpGpVRiLFLT0SHaTz1LD9bji1TcfQFdWK3Vu8yHyyFn8h3RcpVhW9LP3Tyb42mvpW5YUsXfDiMn1WncgjNutLi6Ghvr1y9InynmzVr6mjdp/7akAAJHRvClntT0hU6TI7ram4IhLlx1sYsjeQbAMp4HPyUW3nXzQUVR53Xe2NlxE7Evp/Qsw6bW6MtTgqV90X37zfjnng3zez7mFObvWlXnRImGhLLwqOVfuyid8Wa4Kfj4Yh8Go57t2/g6sUT2LniJ4zoyK2F4+aCer/cFyIW6qTpKphyU9fYUzbC53OLEeqaWaRMwc1VA+AliBKmfyXQcdMLIy5VUXJfIxEAACAASURBVETIc7LWQoe5eHNmHvrW1pr6XbQKAqYfQvglblXqalO0yomazQhBoBfLixFL32osuCM6j3uZGbEP03rVQimRMBT8Uvgj1OvxHTbfSxXlOxnqf9767fUJRXQMk+/g10EQO6FhxuzRZrg+OipPAiR05OlXu1glZaHDAFFkxCMmKhpxacJCLJqclFlIfBWFqFdJyNZeZVjzTO6dEplvHuHGxVM4ceIvXLh+C0/j9CcPK9LjEB0dB/HIkPLNbnRV5aMUg//ml6LZSaIGmQUOW3B5L03XIVoYWVIgPS4a0XHporV3ROWYl4oMxMdEIzZVj82MxstKQOSdyzhz5jLuvcoQiQGturTfKtIRFx2NOLFB4nOUWUh4fg9Xz57E2asP8CaLh5qLtNhXiEvX3ycoY7G3B7fwoq4Vo8XtiF4rM9/gUegFnD5xHKcuXMOtp3HI5JsVnad6mV//tc+3w3sSOoahM4nIvKD9auSXhk+mo05LgISO07re8oZLXehY3uKC15h2YQS3oKEPFuub+qR8hc2t3VQ/6IXa7YBop4qCd0CiNeRGrUcbbqq918y7QhRLot21WrdI6BhGm5ubi48rszlhzB5YKSm6ErkM10FH5U+AhI78fWwzC0nomI4689YUeKqGW1zh/f0xvBLlL6tqUyTh1rpPOTFUGO3Ws4vomd6SI5XIxqOF3P5UbrXxi7CasSPZYJm+ktDJn+OsWTOFqM7K31fmX4DOcDoCJHSczuXWM5iEjhlss59ilT+36B0jeMpURzP/bujTrw8COjRB7YrckJWbC8r1WquRjGxGa45RJDMMU6px+TkNFuGJtvhzDCss0ksSOvljjI6Ohlsh9nvi7VM3/wJ0htMRIKHjdC63nsEkdMxjq0x/gsMLvoJf9WLCf6Z83oGrmxver+2PMSsvItZASot5LUuzVMaNidz2EYXRZq3mRqbS7LH1ekVCxzi2PXr2EL47zGws+iMCYgIkdMQ06HWBCJDQKRA+VWFldgpeRz5AWOgtPI42Nim64O1KqobcJDy9eR03H7/Rn0gsqQ5brzMkdIxje+xYsCB0vv32G+MK0VlOQ4CEjtO42vqGktCxPmNqwbkIkNAxzt85OTko+z67yCSTnKxU6ptqZ1x9dJa8CJDQkZc/7WoNCR274qfGZUiAhI7xTh02fKgQ1bl85bLxBelM2RMgoSN7F9vOQBI6tmNNLTkHARI6xvtZPHz13XcTjS9IZ8qeAAkd2bvYdgaS0LEda2rJOQiQ0DHez9nZ2ShTll3dnOH233//GV+YzpQ1ARI6snavbY0joWNb3tSa/AmQ0DHNx0OGDhaGr65dv2ZaYTpbtgRI6MjWtbY3jISO7ZlTi/ImQELHNP8eOXJYEDrMQoL0RwQYAiR06DqwGAESOhZDSRURARUBEjqmXQhv375FocLsdimtWrU0rTCdLVsCJHRk61rbG0ZCx/bMqUV5EyChY7p/W7ZsoYrqFC5SCGlpaaZXQCVkR4CEjuxcaj+DSOjYjz21LE8CJHRM9+vMmTOE4avjx4+ZXgGVkB0BEjqyc6n9DBILHWYLA2b/GXpIj4F6ewl2PynykfR8xPuE91Vd7zr2+2I7WMsXLpwXhM6kSd87WO+pu9YgQELHGlSdtM4Vv60QfmD4H2h65janVO1QTq/pejDvGmjfvp2T/qqYbjYzzbx4ifdUv0W+9XxMr4BKyI4ACR3ZudR+BqWkpGDcuLHo2KkjPSTMoGmzpoIg9axWlXwlYV8x36XP+n+Gm7du2u+L7YAtd+rcSbjGk5KSHNAC6rIlCZDQsSRNqosIOACB0LBQ4SbA5DPQHxGQGwFmajkfPTx37qzczCN7TCRAQsdEYHQ6EXB0AiR0HN2D1P/8CBw+fEgQOkuWLsnvdDoucwIkdGTuYDKPCGgTIKGjTYTey41AVFSUIHQGDxkkN/PIHhMJkNAxERidTgQcnQAJHUf3IPXfGAIffPi+SuzQjDVjaMn7HBI68vYvWUcE8hAgoZMHCX0gQwJ+/n4qocNM1c/MzJShhWSSsQScWui8iwzG4jm7EZltLK6CnJeDqAM/Y9HhZ6CvXEE4UtmCEnBYoaPMxKv7V3E5LAJJuQWlQOXlTiAw8Adh+CrkRojczSX7DBBwUqGjxNvbqzGoUWtMPhkLW/1mKhLOY7Z/EwxaeRMpSgNeoUNEwIoEHFXopJ4bgg+59YgarXpus++tFV1BVVuRwI4dOwShs2btGiu2RFVLnYD0hI4yE3H/u4Nrfx9D8JGjZjyCceLkaZy/cB0PXmdCl57IjvgDg+rUxRcbnuCdjT2U/XIvxjTwxsD1tm/bxqZScxIl4LBC5/QAvM8JnXrLn5HQkej1JZVuhYeHC0Jn9JhRUukW9cMOBCQjdBQp97Bnzgh09fWAp7ulHjXRZ0W4hphRJl/BnDaeaDT+BOIVdiAOJZIvTkUrz1aYfu5f2KUL9jCb2pQMARI6knEFdcSKBBQKhbBCcpOmja3YElUtdQISEDpKpN78DYNUAscbLVu1g18Hf3TpHICe3bqjV7du8PdRC58azbqgl+pz5pj60bNrV3T3a4VG1dTnMoKpwYSLeMt7Ifc1jo72hqf3OByPs6PEUKbg6tSm8KwzAgf/sdXAGQ+Bnp2dAAkdZ78CnMf+Ztwq4MyWEPTnvATsL3Te3cdi/wboMGAmDkZk6BhqSsTJoTW4KE9DTA1JN+ytrCjsHVobnu410GnCNoS/5QevFEg4+TUauFfD59uj7R72Vrw+gOFeHvAddRSxdtRchmHSUTkSIKEjR6+STboIMNtn8CskM1vU0J9zErC/0AGg5LWILh9khGBqAy5KU30Ijv+r6yTxZ0pkxjxAeEyG5rBQ5h3Ma+kBz8bTECKEeMTlbP36He4vaANP96aYcV0SHbI1AGrPTgRI6NgJPDVrcwITJnwrCJ3Hjx/bvH1qUBoEJCF0DKHIfrwMfnzOTretiDZrpEeB1/u+QC13DwSseYYcQw3a8FjO803o5e6BGn22mWmXDTtLTcmGgMWETm4SnoScx8Ww50jj/lnJfhOCPasWYO7iNdhz+QUy9PwTo0h7jiuHNmPlop8RtGA51u/6G+GJhr+ZqQaTkbMR/+gazp+7ivA4A+tFKFIQEXoe567cx+tMPZ2TjafJkAULFwhCh/a8ct7rQeJCR4FXO3oJycldVpspUrIjsKojExXqga0vzVJK1rlCFK+xv381eLq3waL7tp7/ZR2TqFbpE7CU0Ek9MwgfqGZBVceMOymI3PUlanGzotjhgpoIeqApOpSp9/HnhDYor3GeC3czKo3GI9ciLFm3ADEodBL3w68IV0+bnYjX4waNKeqrX+g5iz6WC4GtW7cKQoeZbk5/zklA4kInGadH1OSETjsse6T5o2msy97dX4C2TFSo+zbESCofRoF/trNCrsWMm7SQoLEOpfMKRMBiQkeIsFTB2GUjUT2PeKmGabezhL4qEs9hapPCwo3H9b1K8GkXgM6taqPSe7zYcUGRpjNwLTWv2DEsdPahfSGujpbbESe0qvki9cxAYYp6g5XPNQ/SO9kROH36lHC9LV6yWHb2kUHGEZC20Em/jil8fk7LeTAv6JGNhwtaqcSS37InME8qGQfTnLNynq1FF0aENZmJW7RksjkIqYyJBCwvdNQixbP/Ipx6HIe4yOs4uGk37vKCRRmP419W5G46bqg7ZheeqTUQlCl3sHaQp3BTqvljKDK07CKhowWE3uZL4P79+8I19cMPk/I9n06QJwFJC52MsOlozuXntJx127yIR+5LbA5ghq0aYdoN7Z9OLadmReLYxnXYtGkLtm3fiT179uPgvr3Ys2M7tm35A5vWr8fOi681cnyynx/HlvUbsWXLNuzatQf79+3D3p3b8ecfG7Fx+wW8Npx2AGSEYXojpn/tsPyx1GSYFh96KwsC1hI6FQft1ptrlvNkMRpxEZ8yfXbqjqxm3cWcupxo+nAQTmtNkiGhI4vLz6ZGJCQkCELni0EDbdo2NSYdAhIWOu9w9+cW3LCVESJFH9N/j2FYdQ94enyKg/F5w+EaxZKD8RknrPQtWtg4MATiCe7JJwaghr4yvpNwXXyyRmPcG2UCDn/mqbKz7+7XmjPFdJ1PnxGBAhKwitCpMhZnkvR9v7IRPr8Od8OphHFX9H0pcvD417rceVUQGKYZ4iShU0DHO2Hx//77D0WKssOl7du3c0ICZDJDQLpCJysci1tx08rr/wBzZ2BnhE5FU0aIGDM0lP0Prh45gIMHDmDf6vFoqSVg/AL/wMk7CRpr8GRH/4XVU/qhHneuV/svMW3uYqxcvRHbD4chId/c53e490tLldBp+N1VDRFFlygRsAYBawgdn8VPNSKdGv1Wvsafbd1YAVPUD/sSNI5qvEk62BFFVJGfYuh2JFnjGAkdDRz0xkgCHlXcVddezVpeRpag0+RGQLJCRzytvEa3xTj81ymcNvQ4dxuxOkZ+Eg72RTVGhHTdhlcmJSK/Q8SGz1FHLHaqf4o1D7WGv3Jj8dcPzVVCpWbP5bgjLFBo7KWiwOvt3dnIVc+9iNX3T7Gx1dF5RCAfAtYQOgb3nsq6h1k11Xk8/AJu+T233a2piEjo5ONYOqyTQPPmzVRCp1TpkjqP04fyJyBRoZONhwtbszd/sdAw9LrJVB1Rn2w8WswsyueBWkNPQWvIP3/vKv7FxelseWEoq/FEHH/NhWmUqQhbFMAOXbWZhvNmbp719uwoeDO2mZ1wnb8pdAYR4AnYXOhkXMO37qYJnULlWiPoNg1d8T6jZ/MJ9Pm0Dzcc6oL0dH3DpubXTyWlT0CaQifzNoKacsNWVZrA378juuh4dG7lrRIxLQbOx6kYHeEcvMWVb9lztHNrjHZN1jNsG1RHQ3R5dVuGu28zEbFlMHwZgVJ/NPa/1NW+ca1k3PiRHV6rPRoXUo0rQ2cRAXMJ2FzoZN3FTC9O6JTogWDNESmjzcgvotNBmF6+DbF6aqXp5XrAyPjj8ePHCULnf//7n4wtJdP0EZCk0Em/PgVNuOhNj/WRBsb+3yH1bbaO/bF4c1NxZoQXK4Zm3DJv1hazRUXyVfzcjhNeXL/qde2JxszrGv2w9oHWcBbfvJHP7+4EoRVTl+cX+MvksJORjdBpRIAjYHOho4jGhmZ8RKcx1kSZNIYs+M2g0Ek6hC7FuDYar4a+JkjoCDid5sUvc38RhM7FSxedxm4yVE1AgkInBee+ZqMwnl7DEGzmcBBrYjJOfF5VJXRaz72Hgqw9nB21C195a4odT/f2CLqcWOCZUtnhS9BBJaD64HC+e3mpnUeviIA5BGwudJCJ0EAP7mZTHD32xxv450S/RQaFTupZDP2IEzqfjIPOiV3KNITM8hFuerRgoH7Wcjqybv06weeHDx+Sk2lki5EEJCd0FK/2YVA1VlA0mXwdaUYaovu0FJwaxGyx4IFWQXcKJHQAJVJvzEcXD7HYqYex+6L1R5x0dyrPp9kPFqIdI3Q8+uNYYp7D9AERsCgB2wsdIDNsMqrxKyf7zsANM77YBoVO7jMsb8hHjT7BxGtauRjKNNxb1QMf8n1wcwEJHYteVpKtbNOmTYLQ2b9/v2T7SR2zHgGJCZ1sRPzeicuH6YAVBV5A7y3Oj2bza5r9FJZnpVVTsebEHMQ4H7HQYV53xIKQVLP+Q+Xbz7w9h53KXmM4/qYcHR4LPVuJgD2EDnKjsa13aeGGU7rVBGwOeQ3R4shQZsQg7OgqTO7bCN6f70ec1gxEg0IHOXi6rLFQv6t7Pyw5+QBvMrKREnkGy4fV4aat82KIhI6VLi/JVSve72r37t2S6x91yPoEpCV0MsIwqwkrJLw+24VX+a5Bkx+gDIT91FglnHzHXsDb/E43cFyReAU/d2D6VgWtu7LTyYWZWN5fYneU+cnI6dcmoSET0alnxAKDBvpIh4iAMQTsInQAKGKPY2L9QmoxwkRXSn2Eyu4fo1wZ0R5YzOdN1+GlViqPYaEDKFMu4DuD09i9MHrzSrTmkpYpomPM1eL45zCbefJLGWzfvt3xDSILTCYgIaGjRELwl+w0a/eGmHzZEqGNXET/EcBGiD49BLPTXzIeYf1nbFJzy0kn8PrdPzg8tgEXeeIiPO2DcF3Prsv5eSX5GLe6st8aPMtvy4j8KqPjRCAfApYSOmkXvxR2IW+2LsqoXDVl2kPsmtodtUqqIyv8TUj1XOITNOv/I7bdTckTJTWmvZyXh/GjX3nhxsbX/X7TL7H+dgqUSYcQwG0g2nxjdD6k6LAcCOzZs0e4HrZs2SIHk8gGEwlIR+jkPMN61Z5UHvAMWIP/mR8g0UCQemYk6jLRknbL8dScOnNicGhsQzYqNGgLIriMZmX6XfzWnc3/4SM7PkO2IVIci9foib43OXixnh2uqznoJMyceauvcvqcCOQhYCmhAyiQHheNl6+TkK01zJSnUe0PlBl4de8iTh7eix079yP4zBXceZ6UT76bse0pkPrsGv46uBs79x3D5cdJGquZ5yS/xstXZvRZ2wZ67xAEDhzYLwidjRs3OkSfqZOWJWBboaN4i6cnN2DxsmOI1hIdaSHTuA08ffDt6X/z/DdnrtnZT3+DPyN0ao7CWVODRIp/cWlWB5XIqd5lIcL4nZi5zuTGHsckbqiNFzsdZlxEolbI3XDf3+LSN+wsMynurm6473TUEQlYTug4ovXUZ2cjwMy04iN7a9etdTbzyV5b7nWlSLyOpX3rcsM9tTBs10v1f2+5Mdjdv7rqWO0BO/XugGyWx95ewfeqBOIOWGVKmCgnHlcX90YtRiR5j8YRPQlDmeFr0L8GN3zFnOteC4P+iDB+hlduFDZ1Zso1wJR8dwA1iwAVIgIaBEjoaOCgNzIncOxYsCB0Vq1eJXNryTxdBGwW0Um7OgmNVEKAFwWNMXLFaTx7m4PE84HsysDeQ7E9UivUo6vXpnym/BfHh9SAp3s1DDxoeP2O7Mfr8FXn9mjfoinqV+X7yT77tv8OfydqxeffhWPN8G5oX0fzXE/32mjXKQDdugRgwOwQw1Pk/z2O4YxQqj4Yx81OIjIFCJ3rLAQePnyId+/yrh6Vn9Bhdny+eeums2AiO2VO4K+/TgpCZ8VvK2RuLZmni4DNhA5yorFnGLcQoIbgYUVC9XYTsTtCc28bXR02/TMFYvd9Di93D9T75qLBmVcZN6ejBde36p41ULtGDXjx6+Z49MJe7fmu6SGY2kgtclRlvLxQy5NdpJAZzqo58KTBPbaYGVeMAPTqtwevTRryMp0ElXAuAnfu3MHHlSth6tQfcerUX0hLYxev0SV0cnNzVeJmydIlYDZBZFaTpT8iIAcCZ878LQgd5vqmP+cjYDuhw7DNfIZDM/pzuTicQGjQC9+vPI2ovP94WswbyrhDGFKd2ZPqBx0bf1qsGTMqysDNGU3h6V4Dgw7EGjVrxYxGqIgTE/i076fCj3yhwm4qEdOtezfhs3r1fcG8L12mlPDZ+x+URWqqqQltTgyZTJc0gQsXzgvX9sJfF0q6r9Q56xCwrdDhbFBmp+CfiEd4FvtWYzaEdUxkak3BxQn14OleH4FXC7KajoV7+PY6fmzA5AB9g3O0x5WF4VJ1DAEmqsMnYhr7TNEcunbkRODylcvCd2DuvLlyMo1sMZKAXYSOkX2z6GnZj5ahk7sHfL46hgStVBuLNmR0ZUok/jUW9dw90HHpI1g4M8noXtCJ8icgjurkJ3YomiP/68HZLLwecl0QOkFBc+RhflYkjm1ch02btmDb9p3Ys2c/Du7biz07tmPblj+waf167Lz4Wj3hB0D28+PYsn4jtmzZhl279mD/vn3Yu3M7/vxjIzZuv4DXMl7DzWmEDpSJODehPjzdO2NVhARkRXYEVjOzrXy+wd//SkJ5yeMHgKzIQ8CUqA5Fc/Lgow8cnEDYzTBB6MyaNdPBreG6nxyMz3TkuvLLnDDPjQNDIN7xLfkEtzCtrnK+8l6V33mEDoDcV4cwuq4H6o05hji7Jv4qkHByPOq718GX+/+x0fCdPL7fZIV5BIyJ6lA0xzy2VEraBMRCf9q0n6TdWWN7l/0Prh45gIMHDmDf6vHsXokiAeMX+AdO3knQuLdkR/+F1VP6qUYRGCHk1f5LTJu7GCtXb8T2w2FIKPCWS8Z23vbnOZXQAXLx+sh4NHZviElnLLcooaluU/x7Bj809ECjsUfwWsYXl6lc6HzrERD/2OsbvqJojvX4U832I3Dt+jUhohP0c5D9OmK1lt8hYsPnqCMSOp7VP8WahxmaLebG4q8f2H0aa/ZcjjtvnWckwcmEDgDlW9z6NQC1G47H0X/soDJyY3FiQmPU6rgQN7VWWta8KukdEbAsgT6f9hF+8LXFDkVzLMuaapMOgdOnTwnX/bLly6TTMUv2RPEvLk5vwy3Iy81objwRx/n/pJWpCFsUgBqMGGozDefj7TqkYUnLjarL+YQOg4XZv+rr5vDpusi2YkOZjrvLe8G36XgceCmBPCGjLhE6SS4EDEV1KJojFy+THdoE9u9X73W1adMm7cPyeZ/1DNsG1dEQO17dluHu20xEbBkMX0bk1B+N/U5473FOocNc2jmv8Ne0ALQbuRfRNsk2z8Wrw1/DL2AaTv5jkwbl8wUmSyxGQFdUh6I5FsNLFUmQwObNm4WIzt69eyXYQ8t1SZl8FT+3Uy9iy+Ti1OvaE40ZkVOjH9Y+0BrOslzTkq7JeYWOyi3ZSIh8hQybDFUqkRkbhQSTdzeX9PVDnXMwArqiOhTNcTAnUndNIvDbyt8EoXPy5AmTyjriydlRu/CVt6bY8XRvj6DLiU67KK2TCx1HvIypz0SgYATEUR2K5hSMJZWWPgFmkUA+J41ZPFD+f0qk3piPLvz2Raok5XoYuy9aY10d+XNQW0hCR82CXhEBpyAgjupQNMcpXO7URv744xRB6Ny9e9cpWOTEHMQ4H+2oTkcsCEmFTQYwJEaZhI7EHOLo3UlKSgLzY0IPaTPo4NdBtb/VlStXyFcSv16fP3/u6D8Ldu3/+PHjBKETGRlp177YonFF4hX83IEROVXQuis7nVxYSND7S+yOcr6JMCR0bHHlOUkbTFi4eIn3hB8VPlxMzy7ExI0YFOR7MGfObCf5FbG8mUOGDha+fwkJCZZvQEo1ZjzC+s+8VDOvWk46gdfv/sHhsQ00ZmJ5tg/C9WTniuuQ0JHSRergfZkxY7rwg1KQH3UqS6KArgHNa8DH19vBfx3s1/3efXoLv0tZWTKeDcIsmzK2oUrU+A7agoh3LHNl+l381r2ahtjxGbINkTJGoX21kdDRJkLvzSbALK/O36D8O/qDGRunBzGga8D8a4D/PtX1rmP299LZC/r5+6l+lwoXKSRfFIp/cWlWB5WYqd5lIcK0FqPNjT2OSU00c3Y6zLiIRCdZN5CEjnwvfZtbJhY6t27fsnn71CARkBuBT9wrq27SJHTM92yTpo1VDD/48H3zK5FyyZx4XF3cG7WY2VXeo3Hkle4V/zPD16B/DbHYqYVBf0SAC/xI2cIC942EToERUgU8ARI6PAl6JgKWIUBCp+Aca9WuqRI6Vap6FLwyidSQ/XgdvurcHu1bNEX9qmLx4gHf9t/h70StHJx34VgzvBva19E819O9Ntp1CkC3LgEYMDsEaRKxz9LdIKFjaaJOXB8JHSd2PpluFQIkdAqO9ePKlVRCR055Thk3p6OFan0cD1T3rIHaNWrAi183x6MX9sZpCZ30EExtpBY5qjJeXqjlWVXI3ak58CRSCo5bkjWQ0JGkWxyzUyR0HNNv1GvpEiChU3DflCpdUiV0WrVqWfDKqAaHJEBCxyHdJs1Ok9CRpl+oV45LgIROwXz37t07lchhkrp79upZsMqotMMSIKHjsK6TXsdJ6EjPJ9QjxyZAQqdg/mMWW+Rnro0eM6pglVFphyVAQsdhXSe9jpPQkZ5PqEeOTYCETsH8d+36NUHozJ49q2CVUWmHJUBCx2FdJ72Ok9CRnk+oR45NgIROwfx38OABQeisXbe2YJVRaYclQELHYV0nvY6T0JGeT6hHjk2AhE7B/Ld6zWpB6Bw+fKhglVFphyVAQsdhXSe9jpPQkZ5PqEeOTYCETsH8N3PmDEHohNwIKVhlVNphCZDQcVjXSa/jJHSk5xPqkWMTIKFTMP+NHPWVIHSioqIKVhmVdlgCJHQc1nXS6zgJHdv6JPfFVgyo5w4Pn75YH5Fj28aRixdbP4dvFXd491sPmzdvY2vt1RwJnYKR79a9myB0ZL2hZ8Ewyb40CR3Zu9h2BpLQMcA6/RYWf9YQVTy80f0Xyyy1nnl7Kqq5MbtcV8a31zMMNG6NQ5m49ZMnexNx/wbXbN28NUySYJ0kdArmFGY1ZGZ6efkK5QpWEZV2aAIkdBzafdLqPAkd/f7IefIrvFWixAWuvr/iiQUCMJYQOorsHGgtFq/fCI0jJHQ0cFjpDQmdgoEtXaaUSugwG3vSn/MSIKHjvL63uOUkdAwgTb2Cn1pWQpmyFdF8ymWkGjjV2EMFEzppOD+youomUMxvM6IVxrbKn0dChydhzWcSOubTTU5OZiOObi7o91k/8yuikg5PgISOw7tQOgaQ0LGtLwomdJIR3PM99kZQfxme5ZradxI6phIz53wSOuZQY8vcu3dPEDqTJn1vfkVU0uEJkNBxeBdKxwASOrb1BQkd2/K2R2skdMynHhx8VBA6K35bYX5FVNLhCZDQcXgXSscASQmd7EjsmTYCXwwZhaBjMcgbsFAi8coSfD1sEEZO34PIbB0cc6JwaM5IDB48EvPPx0NzdCcL0Rc3Yfbo3ujQ1Bt1fBujfc/hmLb2b0Rl6qhLEY8Li8ZiyNDRmHf6jY7+AMqMCAQvmYQv+3VEi0YN0LRdVwwYORrjxo3lHuMQuPIKErmO5BE6OW9wdcdCfD/IH43rN0b7PqMxb/9DpIqT47uHLAAAIABJREFUcJRJuLLie4wf9xW6ujOJzC5wLdsSg8dybYz/HsuvJuowQPuj/CI6CqQ+PoH1P3+Dgd3aoolvHTRo1Rl9R0zAwsNPkS7uE3Lx5vRcjB4yCCOnbccTXfxUzWcifEsghg8egonr7kE7/zkr+gI2zR6J3h0aw6euN5q074ER09bgjG6HIP78AowZOhxTtj1FNjLwdN909G/ji9q+zdBj4nY803VNaGOw8nsSOuYDXrV6lSB0Dh06aH5FVNLhCZDQcXgXSscASQmdnMdY4MvdyBsuzTv9WRmPfQFFuR9CL8y5n/eulh0+F3VUCcQl0PNgvJC0q0y7i1X9qwo/ovymgcKz11Bse5al6ZjMW/ipOtufT769pnWTViLt1hL04IUHn7Ss67nyWFxJZ6sWC53RuzdjdP0iOvtUfdR+xPBKL+M6JnpwXHTV7+aCqoGhmn3X+c6A0Ml5jh1f1UYRPfUznCr3+g33BaWixOsdHVFMdX5ZDDiZrLNFJJ/AwA90MFSm4e7qz+Cpt73qGLbtGTQ9ou5/0S47cWFhO5TUKN8KuxJ0d8OWn5LQMZ/2lCmThe/DzVs3za+ISjo8ARI6Du9C6RggKaGDbDwIqsn+0BVqjvUvNeMxSD2FQR+qb/j1l0VAcyJULiJ/b8SWLx6AvXFcCCI3Bjv7leV+QCugy4ytOHMvGm8iQ3FizUg0eI+r02cGwoQbOQBDQif1EiZU5crV/AIrTt5DdNxL3Dm+GANrurJtfRiAWWs2YtupZ4JIUgsd3o5C8Oo9Bb/v2o8dKwMRUJn/vCz6H+GEmjIN4YfWYNmyeRhVhztesQ9mLlmKpcuWYunytTj6RNxxfdeXWii4ak0vV77ahKaMaChdBz0mLMb245dwJ/werhzfhB87VeDYuaHF6kghsqWMP4Depdn+lOl3BLpiSknH+6OsSoxUweQwPuyTi5idffE+J1IqBkzH1jP3EP0mEqEnVmNUQ17M1sXMm2K7RP0vXxMfqcp/DL8x0zF78gj07BGIS2n6bLfd5yR0zGfdtVtX7lpzQUpKivkVUUmHJ0BCx+FdKB0DpCV0gCxhnZnC8NvxRojIMMQyQibAnbs5MhEGt5YboKGFFK+wpV0h1Q9lEf/teM3pnOQzw1FeVa4Meu+IFm7UvBfSrv+A6qrjJdBzf5y6TQNCJ/nEZ9wN3Bs/P9CMLGWHz0cDVX0fYMgZzblamkKnJDosvKExTJUTsRKtirDi4b2u+6EZoLBiMnJONM4dOItnmuNTLKL0q/iOE3VurbfglTCElYyTg95nb0yleuBAvHCAQ5uI4H6l2eO1ZuIuH55J/hsjKrA2lu2zA9F85EpwyDUE1mCPl+y1H7xeBURCh+FbyBuB5xO1hif5Suz3TELHfPYVK7Giulp1T/MroZKyIEBCRxZulIYRUhM6yAjBd9wwTcleB0U3+myEz62lumkW9fyEHWIp1hG7YtU3V2X8PgQUZ26QhdDmj2juBpiIQz2Lszdbn3kI19QkrBNynmJxPe7GO+AkhP8j9QqdHDxeWIet0zMQQqCCd2lmKAI92frqzAuHuEm10CmMZvPCkKbuPltaGY/93biZVV7T1eJAddSKQofvu87nNFwYWY61t/pU3OIDMwDSLo5ERZWoK4bOuzSFKRIOoU8ZloP3fDWHxEM9UEJVpi7m6XYIni7xYdt7/3P8pXaIesFDt+LwWxOhwVZn1+3wIQkd86C/efOG9TlNLTcPoMxKkdCRmUPtaY7khA7ScXkcu1aM6wcDcIq/ySmisK4Zc9Mshi4b1nOCphQ+DU4S8KmjLM2w5gUXJsi4im+4PJrK4y8hJScHOXke8TjapwT7I9twBSL5CIMBofNkMbt6q2vVSbghuvGrOpN6BoO5Ibb6K9RDPcwxtdDRtzJyJsImV2H78sk4IbeHNdJeQicDNya5s31ynwCNBZ0zQvB9FVbMFPHbilfCaKMS8Qd7cjk0DbD0KT/ImIGr31QW7LuUossfOYg/2psrWx8r1A5RC52Px+Ayl/ckXAASeUFCxzxHnDhxnL0u3Fwwb/488yqhUrIhQEJHNq60vyHSEzpA6vnhKKf6j78CRl1i72bKuL3orMqlaY4NL15iKzdE9dHws9xCfmm4OKo8+0PZeLl6jZmkQ+hSjL0RC4nHqrr1fFY7CA/5EIxeoQOkXxmDyqp6ymNYcJxo+CQXL7f1QGnVsbxiJn+hk4X7c7xYOyqNAmc+d6FYX+goksNxbOUkDO7lh5YNa8K9fBmULl2CSzp2gav7t5pCB1m4O4vrb5FW2MCPJSrjsK8HG0lza74az3nxiCQcDtCdgK3bPzURpHaIWuho5RjZ/5uk7gEJHTULU17NnTdXEDonT54wpSidK0MCJHRk6FR7mSRFoYPkkxjAzdSp8sMNMAGTlNMD2eRV7/l4lJOL56sasz+K/A1PFFnw+fWxOkk5YTfa8sKmkg+atWiOFvoerfwxbHkYhHxWA0IHuVFY34bNB3J1q4C2w77HrKDp+G5ISy5J1gXF/VfnmTkmWaGjTMXNFb01cqBcC5VAOfcqqOrpgUqqIUFdQgfIebwQ9TjGzVY/V+VAKWP3IKAEIyYLo50wjMhc5QnY3ZYXmRXg28yAP1q0hP/wZQhTO4SEjr1+KGzQbt9+fQWhExsba4MWqQkpEyChI2XvOFjfJCl0kIijfdn9blxrzcb97Axcn8AOd/DCJ+fRfG4fqhqYeTcL2fdno5bqZlsTQeLk4ORg9CrJ3ljdvwsRZj8Z5SYDQifjdhAaFWbqLYsPueRhdUSiLJqPWYfQJGEcR2hOqkIn5exX+JgTK5V6z8WRB0mipG3x0JV2RAdAbiRWNeNmmjVehogcJd7s6oz3mPre64QdfFa4ikIyjvXmcqY8JiJEPKlKoKTvhSgZmRe4+k614+cU0TEPvmc1dvmHSh9XNK8CKiUrAiR0ZOVO+xojTaGjRPz+biiuuvH6YvGDu/hZNa36Qww9x81iyrqFadzMHN/FDxG+rB7736DXNNzhZ/cwaLPuYXYtVugU6bhbNIPHCO56hU4SgvuWVLVXa859ZKRG4NKRHdiw7g/sOXEFD19nqmduaTVTYKHTi0tUrrcYQtqLVhv63+oTCqk4M5ibfv/JOB1TtPMROlAgZkt7bg0eHywIj8L2juzwVMme4llTTM+ycG82N9RV1B+71VOq9HdbOKKv/8IJknhBQsd0N4j3uOrWvZvpFVi1hALpcS/x4lUysrUnD1i1XeeunISOc/vfotZLU+gAyjc74Mfl1jSZNRONGdFTojsOCvOtRUmtTWdjDhdRqBIYqhrqUkMSJTcXbYM1kXxSrPoMva/0CZ3cZ1hWn4sSjT+B16ZUKUyfz5u/w/bDUI5OKk4N5ARJ1R8Qqp0ErdcQ/oA+oZCAPe24iIzvorwCShGLQ59z7ebJ0WHrVsbuRTcuclbrm+loU5ThUwb9gvOursPkN1VSidjCaLs2Uj3MyHdT77O+/ustYJcDJHRMx37+/Dlh2GrmzBmmV6CvhDIDca+STLjG8laUfmkkKqiuVxc03/Ay7wkO94kSGXGvkGTC75Y9TCShYw/qMm1TqkIHihj80ZbLgSlaWPUj6NZ6E2JEo0GpZ4dy+TCFuWhCZXxzLe9YSNaDuWjI/VAVaTwJR6PEIR/GsQqkPTuLjXPn4/BL0bdfn9BBJsJ+4hY2ZOotVBYeteqgXqPGaNG6PTr1HoSvpy/BrhuxeX5gCxbRycb9OXy71RB4JYWNHOWmIiFdBEbvtapPKDBisBJ7oylUH0G3RNOZsqNweFIjFOX45U1G5htLRHD/MsLNSjWM99FgnOZnzfGnMc9ZDzC3ESsUXYs2xA9Ho7RWQAYUac9wbuMvmH/4pYihvv6LK7f/axI6pvuAWfiSH/o9ePCA6RXoKKGI2YHe3HpNhX0mmzhMqq4w9QyXH+jmggYrn6sPOOIrRQx29OGWiihSF1Nu5P29lIpZJHSk4gkZ9EOyQge5eLG2qfDjx/wI+ix6IrrpAUg6ik+5dVpUP5KVRuKikLgqdk4mHizz42ZCMTfYwviomg9a+neCX5sm8HHnhoPcPlAPjTHF9Qod5tgDLGrBCjD+B1rXs3u/9Xgi0lUFEzpA5s0p3OKGjB0lULlmDVQu4YrqU41ZLl+/UMh6MB9NVDlHTL2V0arPQAwd2A1NKrKCpFSNyijEiB09ER0GV+pZfrYcW6bi6AvqxG6xO1T4lsJP5Lsi5arCt6UfOvm3RlPfytywpQs+HMbPqlM5hJKRtTjK5e3gIYOE7/qLFy8sYlZGyLf4hBfobm2wW4gGm1a9wwid3HQkJaUhx9DwWsZ1TBBtW9NWJ5RcpCclIc1gRaYxNOdsEjrmUKMyOglIV+gwOa6ruSEQ5sZZG7M0km8AKONx8FNu5V03F1QcdV7vjZVZVTdi30/oWYfNrdEWJYXK+6L795txT7ybZvZ9zKnN3rSrTgkTDYll4dFKP3bRu2JN8NPxcEQ+Dce92zdw9eIJ7FzxE0Z05NbCcXNBvV/uCxELddJ0FUy5qWvsKRvh87nFCHUl3CpTcHPVAHgJooTpXwl03GTMzUEUEfKcrLXQYS7enJmHvrW1pn4XrYKA6YcQfolblbraFK1yossqIwSBXiwvRix9q7Hgjug87mVmxD5M61ULpYSbEV/WBa6FP0K9Ht9h871UUb6Tof7nrd9en1BExzTy//33H/gVkctXKGdaYQNnK+MOY5gnOyT7vv9ShIv+4TBQLM8hhxA6qWcwjIteleh+UOd2LCrDlHE4MrwK3Jjv3Ad+WJoHSirODuciPiW74VDekec8fKz1AQkda5F1wnqlLHQYdygy4hETFY24NGEhFk0vKbOQ+CoKUa+SjEwUVCLzzSPcuHgKJ078hQvXb+FpnP7kYUV6HKKj4yAeGVK+2Y2uqnyUYvDf/FI0O0nUNWaBwxZc3kvTdYgWRpaYxMZoRMeli9beEZVjjUZ8TDRiU/XYzGi8rARE3rmMM2cu496rDJEY0KpL+60iHXHR0YgTGyQ+R5mFhOf3cPXsSZy9+gBvsvh/D3ORFvsKcen6+wRlLPb24BZe1LVitLgd0Wtl5hs8Cr2A0yeO49SFa7j1NA6ZfLOi81Qv8+u/9vl2eE9CxzTozOad/D8ew4YPNa2wDc52CKGTfATd+SUgWvwJ0YLxJhJKxtEexTh/NMef5ldkYrt5Tyehk5cJfWImAakLHTPNsmqxtAsjuAUNfbBY39Qn5Stsbu2m+sEo1G5HAX54rGqKRSvPjVqPNtxUe6+Zd4UolkUbcYDKSOiY5qSgn4MEobN3717TCtvgbBI6NoCsowkSOjqg0EfmESChYzq3zFtT4KkabnGF9/fH8EqUv6yqTZGEW+s+5cRQYbRbzy6iZ3pLjlQiG48WcvtTudXGL8Jqxo5kg2X6SkLHNI7Nmzdj/yEo7GbhHcsVSHseigvnLuNBLL/cub6+5SLxwUlsX7MYv8xdhDW7LyCS2+DWFKGjSHuOK4c2Y+WinxG0YDnW7/ob4YnaPxDiPuQi6cl1XLgYiueije+y34ThyKZlmDf3V/z+53HcjdNTR9YbhN8KRej5ZWitmunoAlffIJwKDUVoGPMIw72oNI2IL9PH0AtncflBrGivuCy8Cb+J0LBzWNaazz30RtCpG1w9oQi7F8XuzafMwMtbF3DmfAieGLSNCT1nIObORZw9H4JnqUJYWwxA72sSOnrR0AFTCZDQMZUYgOynWOXPLXrHCJ4y1dHMvxv69OuDgA5NULsiN2Tl5oJyvdZqJCOb0ZpjFMkMw5RqXI5Ng0V4oud32TGMKVgvSegYzy8hIQFuhdjvS7t2bY0vaMyZqX8Le865NvxNtA2JZuGsZ/sR2O4jIarED6O5FquNwatD8fJU/rOulKn38eeENiivK9/MrTQaj1yLsGQd47GiPtaYcQeZaQ+wdVwTdhV4jbrKo+Ocs4jT0Aq5iFheP2+/NcppTyBIxZkh73Nl6uM3bm+W3IhlaKBdLs/7ypgQkgFF9Ho05Y/VX4hHBjRk2qVR3FISLqgxw7QoLwkdzeuU3hWAAAkd8+Ap05/g8IKv4FedH88WJdK6ueH92v4Ys/IiYg2ktJjXsjRLZdyYyG0fURht1mpuZCrNHluvVyR0jGe7bds24Ua98NeFxhc05szUUxjIbSXjWn+Zev87UdnsZxvRp5z4u1sEH7lX4DaUZT53Q7UW1VGYu7Hrml6uSDyHqU34KAizGngl+LQLQOdWtVFJtT8fW3+RpjNwTTzZgemHqI9VRi/C1w34bWVc4Fq6LLu6OC8q3AqjxeKHoiHhXLxY15pbWkNsg9Zr71miRVRTcfoLbk0sN18se8b+QOW+WCua+KFVXmi/DmYzE0JynmIJvzyEW03Mua8vyzsV50bwAjL/yQkit6hektDRJkLvzSZAQsdsdEJBZXYKXkc+QFjoLTyONjYpWigujxe5SXh68zpuPn6jP5FYHpbmawUJnXwRCScM/GKAIHQePHggfG6RFyIRoVPo5EZhU2d+aQkXVOi1FFe4ISJlxjMc+6Unt3Gv+safR+go43H8y4qcDW6oO2YXnonu+8qUO1g7yFOwseaPoZrb0Ij7yAmKwvVGYkNIHLuURlYMzs5uo56ZWKYPDiWII0NKKHJzkBN/EF35ZOTmmxGTnYOcHPaRKz4duoUOw1upyEVOTjwOdeP/eWuGzTHZQj05QkWMwGrJLjfh5oJqP90UzUgVeS7lbwz5iGOXZ5an6Dw9L0no6AFDH5tOgISO6cyoBBEwRICEjiE66mOpqakoUZIdAq5S1UN9wFKvxCJCR0RHvdSDC1xrTMJVbncZdfMKxJ6ciLpCRCPvgoE5TxajEXe8TJ+dGguaCvVk3cWcutwN/8NBmotoivvo5oIijX/E+QSN8SlA8QZ7e/HLYpTFwNN5OgoYPetKv9Bh+2vcrCvFq23w56NVevasSzn1BT7g2NScfU8UiRLIGHxBQscgHjpoCgESOqbQonOJQP4ESOjkz4g54/dVvwuRjlmzZhpXyJSzxCIij9DJweOFdYX2m615oXuZCOTi+bq2wvCQZkRHtOaVWyWMuyJaUVyjnzl4/CvfVhUEhonWzxL38eMROK6ZhCPUEr+vo7A6eZP10cLnwgsbCx3VGma9efFVCWPz2J6ME/x2NW51MS/cQCKPYITmCxI6mjzoXQEIkNApADwqSgR0ECChowOKjo+8fdibf6HCboiJidFxRgE/EouIPEInUTREUxNBBmYJ6p11pXyNP9uyS0i4FvXDPgMrLycd7MiJpWLodiRZbZjBPqpPS786ThhGq7vwsfoA/8rWQgdA8smBQsSm/JfiFcyZVeuD8dn7XBSr0dK8++fx/TbwTELHABw6ZBoBEjqm8aKziUB+BEjo5EcIuHT5khBN6dW7V/4FzDnDkIjIfYblDbgbcdGOOGBgBWC9QifrHmbV5OoQDW8Js7b0fKax7YKhPopszgz7AVW4+mr9/FB0hHtpB6GDtAsYVYmz/4MBOCnSb4lH+6KMqr+uaL7GvOU1SOjkdTN9YiYBEjpmgqNiREAPARI6esCIPh40+AtB6Jw8eUJ0xIIvDYkI0fYurmX64oSuzWe5rugVOhnX8K1o36j8BA5zvFC51gi6rWfoKk/USc1CkkIHGQj53p3zY2n0PcKrxQQc/pQb1iraFpvUy8KrDTLiFQkdIyDRKcYRIKFjHCc6iwgYS4CEjmFSzNo5RYuxe6p5VqsKpVJjWpDhwqYcNSR0cp5goU9BIzp3MZPf261EDwSLIhpGd9NQH0WVSFPoAFl3Z6ImF2kq0WMf4pTMFoQH0LMUy7ZYp+14baZ7SeiILgB6WTACJHQKxo9KEwFtAiR0tIlovl+0eJEQzVmwcIHmQUu+MyQilLHY0Y7Lr3GrjXkGVr3TG9FRRGNDM37oqjHWRGnNljLGFkN9FJWXqtBBzmP8Wp9j8F5H7Hidi7h93dgNj92Ko/u+OI1VmUUm5fuShE6+iOgEYwmQ0DGWFJ1HBIwjQEJHP6e0tDRhp/IiRQsjPj5e/8kFPWJQRGTixg8enOAqhLabonVvsqtMwdWf6gjCTHPWVSZCA/k6iqPH/njTb+oG+6gGkL/QOYpeqo2GXeDa6He80Ku58p9eHtybX/W9AX7XXxHXuVw8X92M3Q3drTDabbqFnV25dXjK9sVRfjRLbYrRr0joGI2KTsyPAAmd/AjRcSJgGgESOvp5zZgxXRANY8eN0X+iJY7kIyLSLo5EBT5h2CsQ19K0GlWm4d7vXTW2Y9AUOkBm2GRU4+vwnYEb2nVoVZnnbT595M/PV+ikncPw8lxk5ePRuKRvpruBBQPZttJwXljNuCLGXNZbEd81KGK2oAO/z1atLvDmXn8w6CTMGc3jKyahw5Og5wITEAsdJlmuVOmS9JAgg5KlSqBwkUJgpuIyi6yRn6R7nfJJqXW96xT4+ymnCpgp5MVLsCsRly5TCnFxcdY1Lz8RkXUPQb780JMLyvdYgDNRmVAiF6nPTmHZ8LrC+jm8T7WFDnKjsa13aUG8lW41AZtDXmssjqfMiEHY0VWY3LcRvD/fr8pjEQzPr4/cifkKHUUU1grDaCXhv+QaYrMVyHodiuDDN5EoRHjyi+goELWuqWBPqY6LcS02G4qs1wgNPoSb6ooEE6CMw74efBSI51kOX57XsbChulS+r0jo5IuITjCWwNJlS4WLmv8y0zP/ZaVnuhbMvwbatGlt7NfQKc4bOmyI8Fszb/4869ucr4hQIi10Nprw0Qg+MqPx7InhG9aiKzcslEfoAFDEHsfE+qI9qpjypT5CZfePUa6MaA8s5vOm6/BSEB2ae13p3KaCo5Sv0EEuojb6ae2NxV27lUbiohCYyU/oALlRG+HHbyehwaICRukJFSUd668R+XL9ZBzyrCFoosdJ6JgIjE7XTyAxMRHMD1DLli3oIWEGvr4+wk2CGRohf0n7eu3arSuuXb+m/4vnZEdu3ropXL/uHp/g3bt31ieQdgFfVeBu9s3WQneusAJJISsw0KeE0D9e3JdrPhLrbyVDmfo3vuA2B22+UceqxMw+UWkPsWtqd9Ti82Q0BIILXEt8gmb9f8S2uymaeTxG9VFzdpPv4ie62eVE4+CEJiir1XbJThtEO7en4eLIcpytTbFWJ5QcRB/8Fk35Bf/4+kp1xAZut/M8HUj7/+2dd1RU1xbG/3gvienJS6+go2LDCmKs2HuLiSUao4nRWKIxmhhbFGPXRGMsib0hKmIXNTYiYsGuWLBEUAiiCKiAg8zke2tf5l7u9D5M2Xct1szce+rv3OX53OecvQ/ia0XRfwpKDjtqOP6VXkbjN1joGGfDT5iAVxI4Fn9M+od47NgxXtlH7pT3EmjYMFR6f1etWuWyjqoe3kZyUiruKc2dcVYi/fxfiI6KQMTGnYi7nKkVEqIgOw3JKRl4ZK4YdQ5SzsQgetM6rA6PxNY9sTh1/V5hgE4jvbasjWrkZtxE0s0Ms0FzVTlpSDxxADt37ETMqWQ81G2z6iFuJych9Z5SW3Tptk+Vg7TE44jZuR27/jqFZL2CZBlo+aqdKBbLYgxFObfzYqFjJ0DOzgQ8jQALHU8bMW6vSGDTpo2SyKkZEox///1XfMSfXkKg4MZC1C+hsehUngQbQlvpkWCho4eEbzAB7ybAQse7x9dbe5eamoo33xKXSv6DmL9ivLWrPtyvfFyYVkUSs8GzE01asCwFxULHUlKcjgl4CQEWOl4ykD7UDZVKhdDQBtIE+Fmvnj7Uex/qam48RpTRWHOeqoM/bhQ4pPMsdByCkQthAp5DgIWO54wVt7SQgNx1BR21z8nJYTReSCD7QB+8rdmw/HSTFUiRnyqzo78sdOyAx1mZgCcSYKHjiaPmu22mQJ3i6SXy+5SQkOC7MLy65wW4GfEJqpZWoGJIB/wUl2l6g7MVLFjoWAGLkzIBbyDAQscbRtE3+kCOAV9/4zVJ6Cxfvtw3Os69dCgBFjoOxcmFMQH3J8BCx/3HiFsIFBQUoF69upLI+fyL3oyFCdhEgIWOTdg4ExPwXAIsdDx37Hyp5f37fyWJnMpVApGbm+tL3ee+OpAACx0HwuSimIAnEGCh4wmj5LttJN84gwYNlEQOxWa7ePGi7wLhnttNgIWO3Qi5ACbgWQRY6HjWePlSa3VFztPPlMCOHdt9CQH31QkEWOg4ASoXyQTcmQALHXceHd9tG4mcgQMHSJacZ559mkWO774ODu05Cx2H4uTCmID7E2Ch4/5j5GstNCRy6Fg5X0zAEQRY6DiCIpfBBDyIAAsdDxosH2gqiZwBA/prWXJY5PjAwLuwiyx0XAibq2ICriRAE4ihyxKhYyyvofL4HhOwlQAdIe/3VV8tkbNzZ7StxXE+JmCQAAsdg1j4JhPwfALr1q3DzJ9nIisrS6szpoTO48ePsXXrFpDLfb6YgDMJkDPAunXrsMhxJmQuWyDAQodfBCbgpQTI78hbb78pTCTBNYMwbNi32LJlM9ZHrpMmF1oyiD0Ui8lTJqN5i+ago7zkbj8m5oCXUuFuuQMBWpqSezx+8aUXsGvXTndoGrfBCwmw0PHCQeUuMQGRwPQZ0yVRI8YLMvdZv3498NKVSJA/HUmAopDLA3TSu1ilamVcvnzZkdVwWUxAiwALHS0c/IMJeBeBBw8e4LXXX7VK7Ozevcu7IHBv3IJAamoqGjYM1XoXKaxDXl6eW7SPG+G9BFjoeO/Ycs+YgEDgp4k/aU0upiw6ISE12ZrD743DCezZ86e0jErvH0Uh5wCdDsfMBRohwELHCBi+zQS8hQBtRn75fy9ZJHZoIzJfTMBRBGif2OjRo/DEk/+V3r+KlSogISHBUVVwOUzALAEWOmYRcQIm4PkExo4dI000xiw61WtUY2uO5w+1W/SA9niFh4fDz/99rffu0549kJOT4xZt5EbvdFaxAAAgAElEQVT4DgEWOr4z1txTHyZw9+5d6USVMaETGRnpw4S4644iQO4L6tSprSVw6DTfwkULHVUFl8MErCLAQscqXJyYCXguge+//05r8pELnkqBFaFWqz23c9zyYidAm40/69VT7x3r+dmnSElJKfb2cQN8lwALHd8de+65jxFIS0vDs889ozcRkeBZvXq1j9Hg7jqKwKNHjzBx0kQ9i2Ht2h/g6LGjjqqGy2ECNhNgoWMzOs7IBDyPwJAhg/WETtmAMiBX/HwxAWsJrF+/HqUUJbXeqff93hP257AvJmtpcnpnEWCh4yyyXC4TcEMCt27dQomnn9KamJYsWeKGLeUmuSsB8ntD+20qVwnUeo+ee/5ZjB8/DnTSii8m4E4EWOi402hwW5iACwh81b+fNEGVLOWP/Px8F9TKVXg6ARLJ5NVYHrpB3Of1SfduSE5O9vQucvu9lAALHS8dWO4WEzBG4Pr163jyqScEsTN/wXxjyfg+ExAIxB2OQ9duXfBUiSclgSwKnLbt2oKe88UE3JkACx13Hh1uGxNwMAEKCRF/PB4tW7UQQkMcOHCAT8Q4mLE3FEdWPtqgTp6yRVEjfr708ougvV5Xrlzxhq5yH3yAAAsdHxhk7qLvElAqlVi1ahXIUVuFiuX1Ji1x8nr3vXfQvkN7zPx5JsjnDl++SeD27duY8NMEvPPu23rvSpmypTH719m4f/++b8LhXnssARY6Hjt03HAmYJzAzZs3MXLkD3jjzdf1JixR3Bj7fObZp9H78144c+aM8Qr4idcQ+OeffzBv/jw0btJYWtKUvxt0n0KDsJ8lrxlyn+sICx2fG3LusDcToCO9c+fN1fNpQhPXq6+9gmbNm2HEiO8xecpkTJ02FePG/YiPO3+sd0SY0lN8IkpLViG+vItAUlISfpn1C+rVq2tQCJO/pS/79sG5c+e8q+PcG58kwELHJ4edO+2NBOjUS6PGjbQmrhdfegGDBg3E2bNnzXaZ/mc/afIkkB8U+f/oq1StzBOeWXrunyAxMRFTpk5BzZBgrfGVj3VwzSBMmz6Nly/dfzi5hVYQYKFjBSxOygTclcC1a9fgX9JPmsDohMyYMaNt2k9BzgPJTwptOhUnQbIGnTp1yl27z+0yQuD8+fMICxsPEqviWOp+klWHrDs3btwwUgrfZgKeTYCFjmePH7eeCYD+py63wpAjtxMnT9hNhia+0NAG0gRJ/lMssQzZXTEXYDMBije1bt06DB78NcpXKCeNnVzckGsB2ndD+3IoPV9MwNsJsNDx9hHm/nk1gezsbK39NfS/c0eeiqFjxuQrRZwo6XTWvXv3vJqpJ3Xu4sWLWLRokRBMs3QZhTRO4niJn+QNu1XrVli8eDHu3LnjSV3ktjIBuwmw0LEbIRfABIqPgDxadIMG9fHw4UOHN4Y2I7ds1VKaROmoOl+uJ/D48WMhSCa5AOj4YUezJ+ooJAOlI/cCWVlZrm8w18gE3IQACx03GQhuBhOwlkBU1AZJfJDfE0v939AxYZoAaYmL9vZYctFE+d7770r10XFjvpxLgETrn3/uFk7G0VLT8y88J/EXLTXyT1pa7NCxA2bMnIEjR4+AhBFfTIAJACx03OEtyLuGbdPCsPaqi47x5t9A1ISp2HyFg++5w/Db0gYSK+TATZzoduzYbnEx5B9HzDdx0kSL823btlXKRyLJnuvkyZMgXz98QQiCSXuqyPJCsaRIrASUK2vQp404bvSpKF0KPT/7VNg4fuHCBXC0cH6bmIBhAix0DHNx2V119inM7xqE0GHRSCtwVbUq3N03Fi2Cu2Lu8SyoXVUt1+MwAlu2bJZEB/nBseY6Fn9Myjt27BhrsqJN2zZS3pi/YqzKS4nJ0kBlBFau5HMO6MhCQ+yXL18u+CeivU8kVuTixdh38mlUrXpVfP31IKxduxYUYJMvJsAELCPAQscEJ3XubVw5eQh7tm3B1s02/G3djl279yEm7ixScw3IiUeJWNq1Eqp0WYTLeSYa4pRHSiSt/RLBgV2w6JLLK3dKj3ypULm/HGuDKtojdKKjd0gTMwV6tPQiUdS0WVMp75zf5lia1ePS0Qbxw0cOY8mSJRg27Fthf5P86L8xMSO/T0uRxGv06FEg5rzHxuNeA26wGxFgoaM3GCpknYnAhF4tUMPPHwpH/QW0x5zzMkGhzsShsQ1QtupARKer9FrhkhvqTPw14gME1BmJA3eLqQ0u6ah3VUKnZsRJkYIuWnvZI3TkS2bkPZd87pi6aI+J/Ii62G5P9bhL+6Bo6Y/EB512Gj9+nOBBmE40Va1WRQiUKvbRkk9yC9C8RXMMHfqNsAR1KO4QMjMzTSHlZ0yACVhJgIWOHJg6Gyd+7VoocCrVQmiDhmjauBlatWyN9q3boH2rRqguCR8F6jZvU3ifnkl/rdGuVUs0r1MN5aS0JJiqYGiMGAyvAKmb+qKqX2UM3HYbxSkx1FmxGF3TH1V6RSLF9JwlJ8Xfi5HA9u3bJKHz08SfrG6JPUKHKhswoL9U/+nTpw3WT5uVP/iglpROPunTpll3209Cx+j//vtvxB6KFfzQkAO94cOHodsnXUGn2ejoNsUAk/fDmu8lS/kLx7upTLL0kMWHLD98MQEm4HwCLHRkjPPOTkOLag3QfcwGJOYYWGrK2IHeZTVWnurf46iZk7zKG2vRu7w/FGUa45sVCXigKVKVHo2vK/sj4KPVSC52caFC6oaeqOgXiK82pxWr6JINBX81QYDiU4mT7O7du0ykNPzIXqFDvljE+mnSFi8SLxs2RKJ6jWrSczGd/PPDTh+KWRz2SZYmEg4UBoO8AZNlZOfOaGE/C3l5ppNIP/44FkOGDBYCllIbmjRtghpB1fHmW2+YbK+87aa+UwBV6nu79u2EPTi0F4dYO+PIv8PAcUFMwAcIsNDRGmS1yY25OUe+R7DGSlO+xzZkaOU18EOdi1vnEnArR26zycWpn+pC4VcTow+LFh4DeV15K+8MptXzh6LmKBxxaZPykfrnPMyKvALZop4re+6RddGpHHHCNea8LycnR/CYTF6Tdf8iN0RK+QcMHKD3XEyflpZmkA+FghDrpyUXlUqF8PBwYYOxeN/U5xd9vhA2Jf918C/s27dXECS0uZpE0po1a7BixQphWYg8986aPQsTfpogWFf69vsSXbp2Efa81KlTW6iPln7koSpM1WvrM7LkkEWHLDtk4SGrDFl8yAMxWYDIEsSBTw2+KnyTCbgFARY6Fg+DEpd+bijt2Wm/PBm2GGNUKZHoQVahZvNxNd/iyp2cMB9/L2oDhZ8CH62wrV+2NfAOojr4QxE6G4kuOllvWzvdKxf5VKFJm+JZGbooTMP/XnlZEiO2TvCUjyZ03YuOhYtlNgitLxyFFn970ieFQqBlNNpbQ3tsKFo37bkhCxAd16dlOdqT427LbLrjwb+ZABMwTYCFjmk+RU9Vt7Cmnbg5uSkWXLFFpSiROKeZIJbaL0uySSgVNcix31Sp69FN4Q9Fvak45yrzijodkcS0wS+49Mix/fHm0ho2DBWEBlkaDF10/NhRgqN7j0/0qqAo52L5r73+qvRdvOeKT4rKTg4MKwVWRO3aH6BFyxbo3KWzIFbopFPYhDDBGrR06VLBUrRnz5+CV+FLly4J8Z3I4sUXE2ACvkGAhY6l45y5C30CNEKnwc+4aMvEnHcWU+tQGW2x6qZ8OcvSRjgxneoWwgUhF4Jxx13kSJCFjk0DSpO6KCZob4ruRUtJtORDPlcM/XXu/LGUv2ZIsME0lI+sG4aCPl6/fl3K37//V6DN0SQ2xDaZ+6xbt45QNjkrnD5jutBWWqaiU0y0r4WWr2gZi5azdu3aKSwP0SmtpKQk4Zi1oT7rMuDfTIAJMAGRAAsdkYSZz4dxw6X9OfUnnrFpT4ny3FQ0oD0+obNw2e2WavJxbUGhtan2mBNwidRhoWPmrTP8uM+XX0iigoI6WnvZuxmZxIcoZmj/jHiR1US0NonPDX0OHDhAzMKfTIAJMAGnE2ChYxHiHBwfXVOzP6cWwk7YIgMKkLS4pVBGyIijMG04V+LalsVYuHAJlq9YjYiIddgQGYn1EauxesUyLF34BxauikGqfPXs0XXsWPIHFi9ZhlWr12DdukhErg3H6uVLsPiP1YhJkSc23Omc+FGoRUKMlpJcIcRcIHSUV7diye+LsHTZCoSHR2D9+khsWBuBNStXYPniRVj4x2r8pQVSib+3Lyliv3Y9Nqxbi4iVy7Fs4UKE70+BeZKG+Trq7vwF8yWhsXLlSquLtVfoTJk6RaqfrDm6F20yJt8whkQO3atStbJuFv7NBJgAE3AaARY6lqDNO4WJH2iWrYJ+wDHTKsVIiXexvUdZKPxK4uPIdJOnu4BMbPtQ3A9k5LPacByRH2/P3IHupYyk9auK4XHyxIabqE7fiK5CGR2wNtUFS2suEDqZWzpJG8gNO3+sju8Py9lkIrqrwmieGkPjIE9tmKRz78qFCi0xWXvJ81sbAoLqopAToohJSUkxWv3RY0eFo9ZiWvlnRobZM4tGy+UHTIAJMAFrCLDQsYCW8vw0hGqOlQd/G4cHFuTRS5JzDKOCSIiEYJxZi5ASKbFbELVhA6LWzsegEB0BEzocS3ecwl35sS9lMnb9NgKdAzVpFY3QZ8RE/Dx7Phav2Izj6fLEeq0rvJF3GpNqU/6qGHbIBdO5C4SO8lYsthLHDeuwoH8tHQHTEN8v3oHTd+RslEiOnouRnQI1aUujSe8fMHnGbCxYuBJb4u8U+yZyOsr8wovPC2KDfMBYe7TZHqFDvmrEKNp+/u8beZG0b9PpJbk4IsGzcWOUdiL+xQSYABNwEgEWOmbByo+VK9B+xkbs3rnLxN9u7D+RBr2Vn/QofOxPIqI1Vt2yzlqSd3kRPhE3QguCqyw+npugs/xVgLToYahLz0u3x68n75uxGhnouCoF4a0LhVLHtWnW5zdQpMlbLhA6WvXnXcbiLuW0xE75jvOQoGOhK0iLxne1iEMZdJx1SnL0qFVWMf+gTcCihYSiXltz2SN0aNOwWK+11qCEhATQKS461v1pzx7WNJnTMgEmwARsJsBCxxy6R+cwvW7h5G946UP/We0Rh/WsPsqEGWhIIqRMT+zOMlep7nMVMvaPLsyvsSyRw8Gh21I11gU1so9NRxth2akBxuy7Y6OH4/vY92UFQQjYuuFat+Umf7ta6ABQ3T2AseQcUeLojw8Gb0Oqxqijzj6GmS0Ll64ajdqHO9ZpUpPddeRD8v4rCg7y7ksnrSy9bBU6FNeKoo5TvRRNm05B2XJduXIF5PyPvBjzxQSYABNwNgEWOmYI554YhzqaSbFMcCO0aNLUwF8ThApLRiHoMWknbuqZc4AHB79GVSpHd2+NmfqLHitxdUUPVJZN0ApFG8w6dR+5icvQsyJN3tXx1bokfWtSUSFmvuXg2PdBggio3Hc/nO4kuRiEDgFQXlmBnhXkYqc02v98Cg9yE7G8eyWh/8F91yPJwDiaAejSx6LjQBIe06ZPs7juy5cvSyJp6rSpFuejtKK4atuurcX5OCETYAJMoDgJsNAxSf8hjnxXOPEr/FphoQlXxuq8bDxQ6vs0EYvP/vNzVCCRUnMszG7RETPpfqozETe+yDuzYJUIbIUO1WjSDkDneed0lrN0CzD3Ow+nwz4QJvqAbtGw2vBkrnjd58UkdAA1Mg+FobFcNPoFom3b6kLfK3Saj/M6y1m6TXeH37QUJAaapE+y8lh6jRz5AyiUhKnNxPKyKCwERSsnoUN1keM9vpgAE2ACnkCAhY6pUcrai8Gazb0VP9uCdMtXB/RKzdzeBWVoYq09CWft8TysvIGIXuJG2SKrRONxB5FhR/sKG6zEhRkNhMle0XGj+Vheer0E8s7ORBsx8KmWkChqq3zZyOrvioYIi3eEClHixprehVY2eTtDxyPWfpAGyDjnltzKQhGyyZmfoy+KeVWufIBkzZk0eZKjq+DymAATYAJOI8BCxyhaFVLXdUU5YRIMwg9x9i3kZEX3QACVVSsMp+0ROgDU2UcxpWnJQkGimaRr9I1Est0OXpQ4P7WeUG7pj7faJHRUWacR+esUTAwLw4Tx5v5+QM+q/lCUa49hY82lDcOEsAmY8stqxN+1W9EVjro6G8cmNUNpudCp0hcb7Adp9K1y9APamyN30udosUOxnipXCZRETkhITTx+/NjR3eDymAATYAJOI8BCxxhaZSLmNdVYIRrNsjsW0/19fQv31wSNwnF7DRL5t7CxXxUtoUOWkRaTjiDb+OqZsZ7K7ufi1LjCI9jle+1GtuyJU74W29KV2Jt83Irqh+pyoUPfm0zGUftAihW45PP+/fugaN7i/hmKPxUREWF33TF/xaCUoqRULgkeEj58MQEmwAQ8iQALHSOjlRM/BrWFCbA0uq65ZbfvlJxjGq/Dlfojxh4FocrAofGNCq0utVsXHieXJupA9Flzw47NyA9xeGhVoewgVzjGK1aho0JGbBiaEjv/OmgnHCcvWl6r2nsNbrj5ZmT5q6srdkj0kO8aW/bSpKengwJj0skqUTzR0hUtYfHFBJgAE/A0Aix0DI2Y+g629S48Zq2oPgyx9ggTTfkFyUvQWhAknbDpjqFKLbmXgwsLOhduag4ZhujUPNza2F+KwVW436URforLtNEHTia2a7wCN5931fmhDopR6OQkLEBXYS9RLQzfkYq8mxsxUNjUXSR2moTFIdMuC5klY+q4NBSRm+JIieJE/GzWvBmiojaYtMbk5eXhYOxB9Or9mbTBWcz/YacPceeOzS+t4zrIJTEBJsAEbCDAQscAtPwrC9BGYyVpM/+KHRYSWeHZe9C3HE2iDfGrTYGk8nEzSiNqKvbA8kTNRh/1Q5yZ1bZw/49o2anQE6uu2mCOyL+OhcJyXRn0jM6UNd5JX4tJ6OTfjMLA6jQWldBzWaImQKsaD0/PQntFkdBR+FVErxXXHDP+TkJoqNi9e/eA9uqIQkX+SUtRnT7qhOYtmqFuvTro9klXVK9RDU+VeFIv/Usvv4ilS5caqoLvMQEmwAQ8hoDPCh1V9iXs+mMaZm9L1pnI7uPoSE0Az8CB+POug/5Lr7yM3xrSJFoe/fZYayJSISNmHJqQkCnZEtOPZWtbbArSsGNIiPaenfpjEGPtpt37MZpTZg0x2yYxZuV7XwxCR3U3BmHCOJRC66nHdPY0FSBt+zeaJUtR8DTAjwfu2uiA0UoeDkz+6NEjLFu2DEHBNfQEjFz4GPpOoR0ocCfHo3LggHBRTIAJFBsBHxQ6KmTEzUBnwcEenfj5FBFJRceVCm6Go1tpmuQC0D082e69OUUjex+HBhduIG46xxorUT7SY2eik7DMUhlfbUox3KbcC/i9EwUNFSdof1TqugSi4aeoHca/FdxYhFaU32anhsbLNvjExUInPz0WP3coZFSt7ybJG7J223JxYf5HKC/jqCjXFcusAaldYLH/OnzkMMImhIGc/L39zlt6wofiZoWGNsDw4cOwefMmkAdkvpgAE2AC3kLAB4XOfcQNraYlCBRBn2PO7qt4kJ+BmOGFDgKr9lyJ6zas/hh/MdTI2NZTmEADOkch3ZSh6NFFLOzVAo3r1UHtQJ1I2hUb4dvdGdoWHeQhYW5vtKtX6NVXLnYC6zZH6xYt0ebj8Thi5oR8xvbC9pXvvs2mo+XG+27kidOFjhKXfu+NVqH1Ua9mZZSVixe/SmgyZDcydMYh7/w8fNGqPqpopfWHonw9tGzREq1bdkbYYTMgjXTXXW7v3btXEjvffjvUqvAR7tIHbgcTYAJMwFICPih0gPzkCPSuVGT5kAsDhV8pNB68Bom5liK0PJ0qLRKf0B6QwEGmT17lHMfYmpr2lSyN8mUDUF4h+s0piY5rb+sInYc4KnlwpuUtylMOFcqULnRSSJN26U+w06SrYzpxRQKwNLqsTXHNUo3ThU4OTowWl/RKIaBsgMBE9JtTut1a3NYROg+PfI8QSeSUQkCZAFQsWxYBJcX3pQx6RJsEafkLUUwpbY11VUzN5WqZABNgAnYR8EmhQ8Ryr0Rh3MeavTiaiS243RDM3X1DsznVLq6GM6tvY9MntHRSHd/Z6YDQcAV23M2Jx48krsp0Q1SagxzymWuO04WOuQb45nOPFTrqXKScPYSD8Ym4x6trvvnycq+ZgA0EfFboFLJSQ5l1C4kXriLtvmv+5czaPxg1/PwR9O0hvQjnNoyfw7I8iPtOOKZebeBe58e4ElvNQkck4dJPTxU62fs+xWtP/EdYdguae93wXjWXkuTKmAAT8AQCPi50imGIHl3ArMb+UFTog+0mN+q4sG3qu9jdj+JnNcKsBIduTDLTiWzs7xeI8p3CkeIiI5KZBvnEY48VOru74hWN0Kk66yoLHZ94W7mTTMB+Aix07GdoZQlqZOwdgiA/f7Sck6hztN3KohyUXJk4VzhtVX2g/uZcB1VhvBh1PvJZ5Bjn44QnLHScAJWLZAJMwG0JsNApjqEpSMGmLwKhCOyH7beLeZZX3cHO/pWhqNALUbdcs3xXHMi5ziICLHSKWPA3JsAEvJ8AC51iGuOClC0YVM0fwUP26B1xdl2TVMjYMxQ1/aphoDH/PK5rDNfkIgIsdFwEmqthAkzALQiw0Cm2YVDj/vFpaF02CIM22R801JZuFKRtxzdBZdFiSjzu6xyztqU8zuMZBFjoeMY4cSuZABNwDAEWOo7haGMpFL9qEOpWaIWZ8fd1fOPYWKSF2dQPT+HXdoGo0z8SSa7cf2xh+ziZ8wg4TOgU3MOlI/sRE38dDzRCWfnPEaydOwUTZ8zH2oN/I8eIgFY9uI7YjUsxZ/oEhE2ZhT/W/ImEjCIP5YZ6n21yM7IS6RfisH/fISTcNvFCq7KQeGw/9sWeRWqukcYZqpzvMQEm4LEEWOgU+9DlIzV6FNrU/RLrk03/Q++wphbcwuYBDdFm5A6kuKhKh7WdC7KbgKOETvae7nhVOAVVBmNOZeHams9RXnMqqjCGVjmEndMWHerss1gxuD7e1EpXeGT8v0+8hOA+CxBvJGS8SaGTEYnGJTTl1A9HuhFKWkfU5/1tJBXfZgJMwJsIsNBxk9FUpl9DqrH//jq6jepcpN244xYnvhzdNS7PPAGHCR3JwlISX/3SB2X0xEtpjDr5SGqQKmMffqj5lBR+4r/PvoPKoS3RvG4FvPOsKHb+gxIhYxCXrW9tMS101qPhk5oy6qzCbalW7S/Ze7pJR9Srz7mu/ZB/MQEm4JUEWOh45bByp5iAcQKOFzpFIkXReTp2XbyN29cOI2pxBE6LgkWdju2fv60ROU+gUr81uFqkgaDOOoUF3RWSCCo34hhydLrAQkcHCP9kAkzAIgIsdCzCxImYgPcQcJbQebt7BJKNeCjIvzQDQRqLz8sdw3HTkFeFR6cxvpJGNL3WHbt1Qoqx0PGed5B7wgRcSYCFjitpc11MwA0IOEXolPwKe+7pLzcVdleJhMkVNdaad9A/9qERCvm4OK2SJl1JDI/XjqzLQscINr7NBJiASQIsdEzi4YdMwHMJHIw9iOTkZL0OmBM6ubm52LgxSi+f7g258Kg84zKM7mtXp2JFgycKBczTjbH+jm5JRb/vRTVFCcHy8wxab84segBAXp9eCIgM3qOjBYt/MAEmIBFgoSOh4C9MwLsIXLt2Dc89/yzad2iPWbNn4eTJk1CpVDAkdJKSkrBy5Ur0+fILvPX2m/ht7m9mYZgUHvLcj87gx3JF+3gKT2SZ/90gQlsRmayPhY6cOH9nAkxARoCFjgwGf2UC3kbgs149pQ2+JDD+98rLqFipgnTv3ffeQSlFSek3pXn7nbeQl5dnFoVJ4SHPnROHr/3MCxu5AHryjXoIO8lLV3KM/J0JMAHbCLDQsY0b52ICHkHg0qVLWiJGLiaMfZ/580yL+max0Hl0GmMDNELn+bbYqr0iZVFdlMhkfRnr0Ug6Xr4SaUZK5ePlRsDwbSbgxQRY6Hjx4HLXmAAR6Nqti8Vi5/U3XsPDh8Y2C2vzNCk85ElVyVhYS7ToBGP+DUNHruQZDH83Wd+9jWjxjKaO4HkwVgULHcNs+S4T8GYCLHS8eXS5b0wAwNmzZy0WOpMmT7KYmUnhoVVKLo4N99e04Tm0jUy3KdyJyfqy96Ln6xqh835/GDzYpX6AIz9Wlliww0CtQeIfTMBrCbDQ8dqh5Y4xgSICHT/sKE3wxpasXnn1f8jOzi7KZOabSeGhkzc3/juUFj0nVxmDow90Eljw02R9BVcxq4ZoNXofQ+J0rFLqBzgzty1eE9vwxH/AQscC6JyECXgBARY6XjCI3AUmYI7A8RPHzQqd8ePHmStG67lJ4aGVEkBBMlZ2eElqw0t1B2PpkVTInCNDnXMT8Vvm4rtOQQjsEonbOm55TNeXj8u/BEvl/9fvI8yMPod/cpTIurYHsz6rqDm2LoohFjq6Q8S/mYC3EmCh460jy/1iAjoEWrVuVSQEZJYNsvC8+NILuHfvnk4O0z9NCw/9vKq07RhS7UntNrz4Ot7zexdvvCyLgUVtC/kdSTpbeczVp846gG9MHmMPQN+lc1BPs2mZLTr6Y8R3mIA3EmCh442jyn1iAgYIHIo7pC0yZGJn5MgfDOQwfetBzOdSFPJav9+Aji4xmFn94DzW/NAG5V8osqxoLaU9/z5qdR6Blaez9PbxWFJfftImjGj8pl4/Xwn5HH+czIL63ka01AQQ/WCRvjNFg43mm0yACXg0ARY6Hj183HgmYB2Bxk0a64kAciqYnp5uXUFCahUe3k5GUuo9KHWWmcwWps5BypkYRG9ah9Xhkdi6Jxanrt8z7l3ZqvpUyL4ah51REQhfvw0HL96DPARXfmYqklJsaLPZTnECJsAE3JEACx13HBVuExNwEoH9+/fpCZ1vvx3qpNq4WCbABJhA8RNgoVP8Y8AtYGq/VicAAAE3SURBVAIuI/Dvv/+iXr26kth5+pkSSElJcVn9XBETYAJMwNUEWOi4mjjXxwSKmcDOndGS0Bk4cEAxt4arZwJMgAk4lwALHefy5dKZgNsRIKtOzZBgPFXiSVAwT76YABNgAt5MgIWON48u940JGCGwefMmIVK5kcd8mwkwASbgNQRY6HjNUHJHmIDlBNRqNdLSjIW+tLwcTskEmAATcHcCLHTcfYS4fUyACTABJsAEmIDNBFjo2IyOMzIBJsAEmAATYALuToCFjruPELePCTABJsAEmAATsJkACx2b0XFGJsAEmAATYAJMwN0JsNBx9xHi9jEBJsAEmAATYAI2E2ChYzM6zsgEmAATYAJMgAm4OwEWOu4+Qtw+JsAEmAATYAJMwGYCLHRsRscZmQATYAJMgAkwAXcn8H8DrYwhTlx08AAAAABJRU5ErkJggg=="}}},{"metadata":{},"cell_type":"markdown","source":"Now here is a list of good practices I used:\n\n1. **Bottlenecking:** A technique to reduce depth. When two layers with a large depth size are next to each other the model gets huge. Instead introduce an intermediate layer with smaller depth. This intermediate layer should be a 1x1 convolutional layer. I believe this was used by the Google Inception model.\n\n\n2. **Batchnorm after Activation:** Normalize values after non linear activation so during backprop they don't get too high (exploding gradient, goes to infinity/nan) or too low (vanishing gradient, model barely updates). I have seen case where people do activation first then batchnorm afterwards, I'm not entirely sure which is better.\n\n\n3. **Dropout:** To prevent the model's from adapting to each other (bad if one neuron is reliant on another) Two tips I read: Apply after batch normalization and use .1 for convolution layers, .3-.5 for Dense layers.\n\n\n4. **Splitting kernels:** Instead of having one convolutional layer with a 3x3 kernel instead have two layers with 3x1 and 1x3. The latter has less learnable parameters (9 vs 6) and this helps the model train faster but still learn information from the same 3x3 area of the image.\n\n\n5. **Small Kernels:** Instead of having one big 28x28 kernel in one convolutional layer, we instead break it down for each subsequent layer. This has less learnable parameters and if we have enough layers it will have the same effect. The first layer will learn lines, the second can learn curves, the third can learn shapes, until finally the last layer will learn digit shapes.\n\n\n6. **Decrease Width/Height, Increase Depth:** As the network goes deeper, we'll need to downsample to get closer to our output shape (one hot encoding of the digit, [28, 28, 1] into [1, 1, 10]). Now that the image gets smaller we can increase the depth with cost in terms of extra training time.\n\n\n7. **No Flatten Layers:** Keras has a flatten layer where you take the width and height dimension and flatten them to one dimension. They do this to use a Fully Connected layer at the end, which can't take values with a width or height > 1. But if you flatten, the model will likely lose the ability to seperate the x and y dimensions. For images this is important so flattening is generally a bad idea. The work around is to downsample enough times with convolution and pooling so that you end up with a 1x1 feature map (with a large depth) and then you can send it to a Dense layer.\n\n\n8. **Use Depths to the Power of 2:** I heard there was some keras speedups that happens if you do this because of matrices. I don't know if it's true but I consider it a good practice.\n\n\n9. **Using Same Padding for Convolutions:** The idea of padding is to zero pad an input before convolution so the input size is the same as the output size (for that layer). It was used by the VGG model.\n\n\n10. **Almost Always Use LeakyRelu:** Activations generally have two problems. Their values are bounded or they are not evenly distributed along the positive and negative sides (For more information check out the course below). Relu is a good solution to the first but not the second because of the dying relu problem. LeakyRelu fixes this, it doesn't just zero out negative values but instead scales them down."},{"metadata":{},"cell_type":"markdown","source":"A lot of these tips I learned from the Stanford Course CSE 231n.\n\nHere is a good link to lecture 2: https://www.youtube.com/watch?v=8inugqHkfvE&ab_channel=AndrejKarpathy"},{"metadata":{"trusted":true},"cell_type":"code","source":"from keras.layers import LeakyReLU\n    \n#Kernel regularizer has a weight of 0 so it's not being used, did not give good results\nclass ResModule(keras.layers.Layer):\n    def __init__(self, input_dim, bottleneck_size):\n        super(ResModule, self).__init__()\n        self.input_dim = input_dim\n        self.bottleneck_size = bottleneck_size\n        self.conv1_height = layers.Conv2D(self.input_dim, kernel_size=(1, 3), kernel_regularizer=tf.keras.regularizers.L1L2(.0), padding=\"same\", activation=LeakyReLU())\n        self.conv1_width = layers.Conv2D(self.input_dim, kernel_size=(3, 1), kernel_regularizer=tf.keras.regularizers.L1L2(.0), padding=\"same\", activation=LeakyReLU())\n        self.conv2_height = layers.Conv2D(self.input_dim, kernel_size=(1, 3), kernel_regularizer=tf.keras.regularizers.L1L2(.0), padding=\"same\", activation=LeakyReLU())\n        self.conv2_width = layers.Conv2D(self.input_dim, kernel_size=(3, 1), kernel_regularizer=tf.keras.regularizers.L1L2(.0), padding=\"same\", activation=LeakyReLU())\n\n        self.bottleneck1 = layers.Conv2D(self.bottleneck_size, kernel_size=1, kernel_regularizer=tf.keras.regularizers.L1L2(.0), padding=\"same\", activation=LeakyReLU())\n        self.bottleneck2 = layers.Conv2D(self.bottleneck_size, kernel_size=1, kernel_regularizer=tf.keras.regularizers.L1L2(.0), padding=\"same\", activation=LeakyReLU())\n        self.bottleneck3 = layers.Conv2D(self.bottleneck_size, kernel_size=1, kernel_regularizer=tf.keras.regularizers.L1L2(.0), padding=\"same\", activation=LeakyReLU())\n        self.bottleneck4 = layers.Conv2D(self.bottleneck_size, kernel_size=1, kernel_regularizer=tf.keras.regularizers.L1L2(.0), padding=\"same\", activation=LeakyReLU())\n\n        self.norm1 = layers.BatchNormalization()\n        self.norm2 = layers.BatchNormalization()\n        self.norm3 = layers.BatchNormalization()\n        self.norm4 = layers.BatchNormalization()\n        self.norm5 = layers.BatchNormalization()\n        self.norm6 = layers.BatchNormalization()\n        self.norm7 = layers.BatchNormalization()\n        self.norm8 = layers.BatchNormalization()\n        self.norm9 = layers.BatchNormalization()\n        self.norm10 = layers.BatchNormalization()\n        self.norm11 = layers.BatchNormalization()\n        \n        self.dropout1 = layers.Dropout(.1)\n        self.dropout2 = layers.Dropout(.1)\n        self.dropout3 = layers.Dropout(.1)\n        self.dropout4 = layers.Dropout(.1)\n        self.dropout5 = layers.Dropout(.1)\n        self.dropout6 = layers.Dropout(.1)\n        self.dropout7 = layers.Dropout(.1)\n        self.dropout8 = layers.Dropout(.1)\n        \n\n    def call(self, inputs):\n        x = self.bottleneck1(inputs)\n        x = self.norm1(x)\n        x = self.dropout1(x)\n        \n        x = self.conv1_height(x)\n        x = self.norm2(x)\n        x = self.dropout2(x)\n        \n        x = self.bottleneck2(x)\n        x = self.norm3(x)\n        x = self.dropout3(x)\n        \n        x = self.conv1_width(x)\n        x = self.norm4(x)\n        x = self.dropout4(x)\n        \n\n        xMiddle = layers.add([inputs, x])\n        xMiddle = self.norm5(xMiddle)\n        \n        x = self.bottleneck3(xMiddle)\n        x = self.norm6(x)\n        x = self.dropout5(x)\n        \n        x = self.conv2_height(x)\n        x = self.norm7(x)\n        x = self.dropout6(x)\n        \n        x = self.bottleneck4(x)\n        x = self.norm8(x)\n        x = self.dropout7(x)\n        \n        x = self.conv2_width(x)\n        x = self.norm9(x)\n        x = self.dropout8(x)\n\n        x = layers.add([xMiddle, x])\n        x = self.norm10(x)\n        \n        x = layers.add([inputs, x])\n        x = self.norm11(x)\n        \n        return x\n    \n    def get_config(self):\n\n        config = super().get_config().copy()\n        config.update({\n            'bottleneck_size': self.bottleneck_size,\n            'input_dim': self.input_dim\n        })\n        return config\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"> Note: I defined the tpu_strategy here and created the model under it's scope, now when I train the model it'll go on the tpu."},{"metadata":{"trusted":true},"cell_type":"code","source":"cluster_resolver = tf.distribute.cluster_resolver.TPUClusterResolver(\n    tpu=os.environ[\"TPU_NAME\"])\ntf.config.experimental_connect_to_cluster(cluster_resolver)\ntf.tpu.experimental.initialize_tpu_system(cluster_resolver)\ntpu_strategy = tf.distribute.TPUStrategy(cluster_resolver)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"input_shape = [28, 28, 1]\nunit_size = 1024\nbottleneck_size = 64\n\nwith tpu_strategy.scope():\n    model = keras.Sequential([\n        \n        #tf.keras.layers.experimental.preprocessing.Rescaling(1/127.5, offset=-1, input_shape=input_shape),\n        # rescales but not MEAN CENTERED! \n        # No longer using\n        \n        \n        # An outside convolution is applied before each new ResModule with a new Depth, to \n        # ensure depth is equal throughout the module\n        layers.Conv2D(unit_size, kernel_regularizer=tf.keras.regularizers.L1L2(.0), input_shape=input_shape, kernel_size=(1, 3), padding=\"same\", activation=LeakyReLU()),\n        layers.BatchNormalization(),\n        layers.Dropout(.1),\n        \n    \n        ResModule(unit_size, bottleneck_size),\n        ResModule(unit_size, bottleneck_size),\n    \n        layers.MaxPooling2D(2),\n    \n        layers.Conv2D(unit_size*2, kernel_regularizer=tf.keras.regularizers.L1L2(.0), kernel_size=(1, 3), padding=\"same\", activation=LeakyReLU()),\n        layers.BatchNormalization(),\n        layers.Dropout(.1),\n        ResModule(unit_size*2, bottleneck_size),\n        ResModule(unit_size*2, bottleneck_size),\n    \n        layers.MaxPooling2D(2),\n    \n        ResModule(unit_size*2, bottleneck_size),\n        ResModule(unit_size*2, bottleneck_size),\n    \n        layers.MaxPooling2D(2),\n    \n        layers.Conv2D(unit_size*4, kernel_regularizer=tf.keras.regularizers.L1L2(.0), kernel_size=(1, 3), padding=\"same\", activation=LeakyReLU()),\n        layers.BatchNormalization(),\n        layers.Dropout(.1),\n        ResModule(unit_size*4, bottleneck_size*2),\n        ResModule(unit_size*4, bottleneck_size*2),\n        \n        layers.Conv2D(unit_size*8, kernel_regularizer=tf.keras.regularizers.L1L2(.0), kernel_size=(1, 3), padding=\"same\", activation=LeakyReLU()),\n        layers.BatchNormalization(),\n        layers.Dropout(.1),\n        ResModule(unit_size*8, bottleneck_size*4),\n        ResModule(unit_size*8, bottleneck_size*4),\n    \n        layers.MaxPooling2D(2),\n              \n        layers.Dense(units=unit_size*16, activation=LeakyReLU(), kernel_regularizer=tf.keras.regularizers.L1L2(.0)),\n        layers.BatchNormalization(),\n        layers.Dropout(.3),\n        \n        layers.Dense(10) #USING LOGITS\n    ])\n    \n    #Removed: kernel_regularizer=tf.keras.regularizers.L1L2(.001)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now you can look at the model for a more detailed look:"},{"metadata":{"trusted":true},"cell_type":"code","source":"model.summary()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Load the model"},{"metadata":{},"cell_type":"markdown","source":"#### Note: ONLY IF YOU PREVIOUSLY SAVED THE MODEL"},{"metadata":{"trusted":true},"cell_type":"code","source":"#Need to pass in the custom objects to work!\nif os.path.isdir('./mymodel'):\n    model = keras.models.load_model(\"./mymodel.h5\", custom_objects={'LeakyReLU': layers.LeakyReLU, 'ResModule':ResModule})","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Creating the Tensorflow Dataset"},{"metadata":{},"cell_type":"markdown","source":"When training models on the tpu you should apply these to your tf dataset:\n1. Cache\n2. Shuffle\n3. Repeat (For most cases)\n4. Batch "},{"metadata":{},"cell_type":"markdown","source":"> Note: For perfect shuffling, a buffer size greater than or equal to the full size of the dataset is required. (Taken from tensorflow website)"},{"metadata":{"trusted":true},"cell_type":"code","source":"X_train_tensor = np.reshape(X.values, (-1, 28, 28, 1))\nY_train_tensor = np.reshape(y_df.values, (-1, 1, 1, 10))\nX_train_tensor = tf.dtypes.cast(X_train_tensor, tf.float64)\nY_train_tensor = tf.dtypes.cast(Y_train_tensor, tf.float64)\n                     \n\ndef train_input_fn(x, y, batch_size=512):\n    dataset = tf.data.Dataset.from_tensor_slices((x, y))\n    \n    dataset = dataset.cache()\n    dataset = dataset.shuffle(42000)#, reshuffle_each_iteration=True)\n    \n    #Take 20% to the validation set\n    validation_set = dataset.take(tf.cast(X_train_tensor.shape[0]*.2, tf.int64))\n    dataset = dataset.skip(tf.cast(X_train_tensor.shape[0]*.2, tf.int64))\n    \n    dataset = dataset.repeat()\n    dataset = dataset.batch(batch_size)\n    \n    validation_set = validation_set.repeat()\n    validation_set = validation_set.batch(batch_size)\n    \n    \n    return dataset, validation_set\n    \nbig_dataset, val_dataset = train_input_fn(X_train_tensor, Y_train_tensor)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Data Augmentation"},{"metadata":{},"cell_type":"markdown","source":"Why use augmentation?\n\nI believed my training set might be missing some images the test set may have, so these augmentations will mess up the training set to prepare my model for the upcoming test set.\n\n\nI used these augmentations:\n1. Rotated Digits\n2. Zooming Out on Digits\n3. Zooming In on Digits (the negative width and height factor)\n4. Shifting Image in any direction"},{"metadata":{"trusted":true},"cell_type":"code","source":"data_augmentation = keras.Sequential(\n    [\n        # I don't think flipping is unlikely\n        #layers.experimental.preprocessing.RandomFlip(\"horizontal\"), Unlikely numbers would be flipped\n        #layers.experimental.preprocessing.RandomFlip(\"vertical\"), Unlikely upside down number\n        layers.experimental.preprocessing.RandomRotation(factor=0.1),\n        \n        #upper and lower bounds\n        #Zoomed out\n        layers.experimental.preprocessing.RandomZoom(\n            height_factor=(.1, .3), width_factor=(.1, .3)\n        ),\n        #Zoomed In\n        layers.experimental.preprocessing.RandomZoom(\n            height_factor=(-.3, -.1), width_factor=(-.3, -.1)\n        ),\n        layers.experimental.preprocessing.RandomTranslation(\n            height_factor=(-.2, .2), width_factor=(-.2, .2)\n        )\n    ],\n    name=\"data_augmentation\"\n)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Data Normalization"},{"metadata":{},"cell_type":"markdown","source":"Why use normalization?\n\nTwo reasons:\n1. The model will have trouble learning if values have a very high scalar value or a low scalar value.\n2. The model learns faster when it's normalized from -1 to 1 (balances along positive and negative sides).\n\nA mistake I made previously was to use batch normalization instead of normalization all my data. It's bad because the same data will change depending on the batch it's with and that doesn't make sense because the image hasn't changed. So I removed this because it was confusing my model."},{"metadata":{"trusted":true},"cell_type":"code","source":"data_normalization = keras.Sequential(\n    [\n        layers.experimental.preprocessing.Normalization(input_shape=input_shape)\n    ],\n    name=\"data_normalization\"\n)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Apply to the training and validation datasets\naug_ds = big_dataset.map(\n  lambda x, y: (data_augmentation(x, training=True), y))\naug_ds = aug_ds.map(\n  lambda x, y: (data_normalization(x, training=True), y))\n\naug_valid_ds = val_dataset.map(\n  lambda x, y: (data_augmentation(x, training=False), y))\naug_valid_ds = aug_valid_ds.map(\n  lambda x, y: (data_normalization(x, training=False), y))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Compile the model"},{"metadata":{},"cell_type":"markdown","source":"This will apply the loss and optimizer for our model.\n\nI found better results using SGD with a learning rate scheduler rather than an adaptive optimizer like adam. I've read that it's because the former generalizes better."},{"metadata":{"trusted":true},"cell_type":"code","source":"model.compile(\n    optimizer=keras.optimizers.SGD(learning_rate=0.1),\n    loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True), #'categorical_crossentropy' #normal categorical is for one-hot\n    steps_per_execution=64,\n    metrics=['accuracy']\n)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### And Now to Train the Model"},{"metadata":{"trusted":true},"cell_type":"code","source":"#This is what was recommended for batch size but I ignored it:\n#BATCH_SIZE = 16 * tpu_strategy.num_replicas_in_sync\n\ndef scheduler(epoch, lr):\n    if epoch < 3:  #10 before!\n        return lr\n    else:\n        return lr * tf.math.exp(-0.1)\n    \n    \nhistory = model.fit(\n    aug_ds,\n    validation_data=aug_valid_ds,\n    steps_per_epoch = 2048,\n    validation_steps = 256,\n    validation_freq = 2,\n    epochs=30,\n    callbacks=[tf.keras.callbacks.LearningRateScheduler(scheduler), ]\n)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#My Notes:\n\n#l1 punishes insignificant weights\n#l2 punishes weights that are too high\n\n#Regularization Weights:\n#For convolution, these values should be more lower (.001, dropout=.5)\n#For FC (.1) also dropout\n\n\n#Recent Changes: \n#smaller batch size, \n#learning rate scheduler, \n#smaller learning rate, & SGD!!!\n#steps per execution","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Saving the Model:"},{"metadata":{"trusted":true},"cell_type":"code","source":"model.save('mymodel64.h5', save_format='h5')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Create Predictions:"},{"metadata":{"trusted":true},"cell_type":"code","source":"test_df = pd.read_csv('/kaggle/input/digit-recognizer/test.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"predictions = model.predict(\n    data_normalization(\n        np.reshape(test_df.values, (-1, 28, 28, 1)\n    ), training=False)\n)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"predictions.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"str_csv = \"ImageId,Label\\n\"\nimage_id = 1\nfor p in predictions:\n    str_csv = str_csv + \"{},{}\\n\".format(image_id, np.argmax(p[0][0]))\n    image_id += 1\n    \nwith open('submission.csv', \"w\") as fh:\n    fh.write(str_csv)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}